| 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373 |
- <!DOCTYPE html><head><meta charset="utf-8" /><meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no" /><meta name="keywords" content="GNU, Emacs, Libre Software, Hurd, Guile, Guix" /><meta name="description" content="GNUcode.me is a website focusing on libre software projects, especially the GNU project." /><link type="application/atom+xml" rel="alternate" title="GNUcode.me -- Feed" href="/feed.xml" /><a rel="me" href="https://fosstodon.org/@thegnuguy"></a><link type="text/css" href="css/footer.min.css" rel="stylesheet"></link><link type="text/css" href="css/header.min.css" rel="stylesheet"></link><link type="text/css" href="css/main.min.css" rel="stylesheet"></link><title>Building Toolchains with Guix — GNUcode.me</title></head><body><header><nav><ul><li><a href="index.html">GNUcode.me</a></li><li><a href="services.html">Services</a></li><li><a href="about.html">About</a></li><li><a href="business-ideas.html">Business-ideas</a></li></ul></nav></header><h1>Building Toolchains with Guix</h1><main><section class="basic-section-padding"><article><h3>by Mitchell Schmeisser <mitchellschmeisser@librem.one> — February 23, 2023</h3><div><p>Today's post is a guest post from my new internet friend Mitchell. We met on the
- #guix irc channel, and I offered to post a few of his blog posts on this blog. Without further ado,
- here is Michell's first blog post (it's pretty fantastic)!</p><h1>Overview</h1><p>In order to deploy embedded software using Guix we first need to teach Guix
- how to build it. Since Guix bootstraps everything this means we must teach Guix
- how to build our toolchain.</p><p>The <a href="https://zephyrproject.org">Zephyr Project</a> uses its own fork of GCC with custom configs for
- the architectures supported by the project.</p><h1>Anatomy of a toolchain</h1><p>Toolchains are responsible for taking high level descriptions of programs and
- lowering them down to a series of equivalent machine instructions. This process
- involves more than just a compiler. The compiler uses the <code>binutils</code> to
- manipulate it’s internal representation down to a given architecture. It
- also needs the C standard library as well as a few other libraries needed for
- some compiler optimizations.</p><p>The C library provides the interface to the underlying kernel. System calls like <code>write</code>
- and <code>read</code> are provided by <code>Glibc</code> on most Linux distributions.</p><p>In embedded systems smaller implementations like <code>newlib</code> and <code>newlib-nano</code> are used.</p><h1>Bootstrapping a Toolchain</h1><p>In order to compile GCC we need a C library that’s been compiled for
- our target architecture. How can we cross compile our C library if we
- need our C library to build a cross compiler? The solution is to build
- a simpler compiler that doesn’t require the C library to function.
- It will not be capable of as many optimizations and it will be very slow,
- however it will be able to build the C libraries as well as the complete version
- of GCC.</p><p>In order to build the simpler compiler we need to compile the <code>binutils</code> to
- work with our target architecture.
- The <code>binutils</code> can be bootstrapped with our host GCC and have no target dependencies.</p><p><a href="https://crosstool-ng.github.io/docs/toolchain-construction/">For more information read this.</a></p><p>Doesn’t sound so bad right? It isn’t… in theory.
- However internet forums since time immemorial have been
- littered with the laments of those who came before.
- From incorrect versions of <code>ISL</code> to the wrong C library being linked
- or the host linker being used, etc.
- The one commonality between all of these issues is the environment.
- Building GCC is difficult because isolating build environments is hard.</p><p>In fact as of <code>v0.14.2</code> the zephyr SDK repository took down the build
- instructions and posted a sign that read “Building this is too
- complicated, don’t worry about it.” (I’m paraphrasing, but
- <a href="https://github.com/zephyrproject-rtos/sdk-ng/tree/v0.14.2#build-process">not by
- much</a>.)</p><p>We will neatly side step all of these problems and not
- risk destroying or polluting our host system with garbage
- by using Guix to manage our environments for us.</p><p>Our toolchain only requires the first pass compiler because
- newlib(-nano) is statically linked and introduced to the toolchain
- by normal package composition.</p><h1>Defining the Packages</h1><p>All of the base packages are defined in <code>zephyr/packages/zephyr.scm</code>.
- Zephyr modules are defined in <code>zephyr/packages/zephyr-xyz.scm</code>, following
- the pattern of other module systems implemented by Guix.</p><h2>Binutils</h2><p>First thing we need to build is the <code>arm-zephyr-eabi</code> binutils.
- This is very easy in Guix.</p><pre><code>(define-module (zephyr packages zephyr)
- #:use-module (guix packages)
- (define-public arm-zephyr-eabi-binutils
- (let ((xbinutils (cross-binutils "arm-zephyr-eabi")))
- (package
- (inherit xbinutils)
- (name "arm-zephyr-eabi-binutils")
- (version "2.38")
- (source
- (origin (method git-fetch)
- (uri (git-reference
- (url "https://github.com/zephyrproject-rtos/binutils-gdb")
- (commit "6a1be1a6a571957fea8b130e4ca2dcc65e753469")))
- (file-name (git-file-name name version))
- (sha256 (base32 "0ylnl48jj5jk3jrmvfx5zf8byvwg7g7my7jwwyqw3a95qcyh0isr"))))
- (arguments
- `(#:tests? #f
- ,@(substitute-keyword-arguments (package-arguments xbinutils)
- ((#:configure-flags flags)
- `(cons "--program-prefix=arm-zephyr-eabi-" ,flags)))))
- (native-inputs
- (append
- (list texinfo
- bison
- flex
- gmp
- dejagnu)
- (package-native-inputs xbinutils)))
- (home-page "https://zephyrproject.org")
- (synopsis "binutils for zephyr RTOS"))))</code></pre><p>The function <code>cross-binutils</code> returns a package which has been
- configured for the given gnu triplet. We simply inherit that package
- and replace the source.
- The zephyr build system expects the binutils to be prefixed with
- <code>arm-zephyr-eabi-</code> which is accomplished by adding another flag to the
- <code>#:configure-flags</code> argument.</p><p>We can test our package definition using the <code>-L</code> flag with <code>guix build</code>
- to add our packages.</p><pre><code>guix build -L guix-zephyr zephyr-binutils
- /gnu/store/a947nb4rb2vymz2gaqnafgm1bsq4ipqp-zephyr-binutils-2.38</code></pre><p>This directory contains the results of <code>make install</code>.</p><h2>GCC sans libc</h2><p>This one is a bit more involved. Don’t be afraid!
- This version of GCC wants ISL version 0.15. It’s easy enough
- to make that happen. Inherit the current version of ISL and swap
- out the source and update the version. For most packages the build process doesn’t
- change that much between versions.</p><pre><code>(define-public isl-0.15
- (package
- (inherit isl)
- (version "0.15")
- (source (origin
- (method url-fetch)
- (uri (list (string-append "mirror://sourceforge/libisl/isl-"
- version ".tar.gz")))
- (sha256
- (base32
- "11vrpznpdh7w8jp4wm4i8zqhzq2h7nix71xfdddp8xnzhz26gyq2"))))))</code></pre><p>Like the binutils, there is a function for creating cross-gcc packages. This one
- accepts keywords specifying which binutils and libc to use. If libc isn’t
- given (like here), gcc is configured with many options disabled to facilitate
- being built without libc. Therefore we need to add the extra options we want (I
- got them from the SDK configuration scripts on the <a href="https://github.com/zephyrproject-rtos/sdk-ng">sdk
- github</a> as well as the commits to
- use for each of the tools. ).</p><pre><code>(define-public gcc-arm-zephyr-eabi-12
- (let ((xgcc (cross-gcc "arm-zephyr-eabi"
- #:xbinutils zephyr-binutils)))
- (package
- (inherit xgcc)
- (version "12.1.0")
- (source (origin (method git-fetch)
- (uri (git-reference
- (url "https://github.com/zephyrproject-rtos/gcc")
- (commit "0218469df050c33479a1d5be3e5239ac0eb351bf")))
- (file-name (git-file-name (package-name xgcc) version))
- (sha256
- (base32 "1s409qmidlvzaw1ns6jaanigh3azcxisjplzwn7j2n3s33b76zjk"))
- (patches
- (search-patches "gcc-12-cross-environment-variables.patch"
- "gcc-cross-gxx-include-dir.patch"))))
- (native-inputs
- (modify-inputs (package-native-inputs xgcc)
- ;; Get rid of stock ISL
- (delete "isl")
- ;; Add additional dependencies that xgcc doesn't have
- ;; including our special ISL
- (prepend flex
- perl
- python-3
- gmp
- isl-0.15
- texinfo
- python
- mpc
- mpfr
- zlib)))
- (arguments
- (substitute-keyword-arguments (package-arguments xgcc)
- ((#:phases phases)
- `(modify-phases ,phases
- (add-after 'unpack 'fix-genmultilib
- (lambda _
- (substitute* "gcc/genmultilib"
- (("#!/bin/sh") (string-append "#!" (which "sh"))))
- #t))
- (add-after 'set-paths 'augment-CPLUS_INCLUDE_PATH
- (lambda* (#:key inputs #:allow-other-keys)
- (let ((gcc (assoc-ref inputs "gcc")))
- ;; Remove the default compiler from CPLUS_INCLUDE_PATH to
- ;; prevent header conflict with the GCC from native-inputs.
- (setenv "CPLUS_INCLUDE_PATH"
- (string-join
- (delete (string-append gcc "/include/c++")
- (string-split (getenv "CPLUS_INCLUDE_PATH")
- #\:))
- ":"))
- (format #t
- "environment variable `CPLUS_INCLUDE_PATH' changed to ~a~%"
- (getenv "CPLUS_INCLUDE_PATH"))
- #t)))))
- ((#:configure-flags flags)
- ;; The configure flags are largely identical to the flags used by the
- ;; "GCC ARM embedded" project.
- `(append (list "--enable-multilib"
- "--with-newlib"
- "--with-multilib-list=rmprofile"
- "--with-host-libstdcxx=-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic -lm"
- "--enable-plugins"
- "--disable-decimal-float"
- "--disable-libffi"
- "--disable-libgomp"
- "--disable-libmudflap"
- "--disable-libquadmath"
- "--disable-libssp"
- "--disable-libstdcxx-pch"
- "--disable-nls"
- "--disable-shared"
- "--disable-threads"
- "--disable-tls"
- "--with-gnu-ld"
- "--with-gnu-as"
- "--enable-initfini-array")
- (delete "--disable-multilib" ,flags)))))
- (native-search-paths
- (list (search-path-specification
- (variable "CROSS_C_INCLUDE_PATH")
- (files '("arm-zephyr-eabi/include")))
- (search-path-specification
- (variable "CROSS_CPLUS_INCLUDE_PATH")
- (files '("arm-zephyr-eabi/include"
- "arm-zephyr-eabi/c++"
- "arm-zephyr-eabi/c++/arm-zephyr-eabi")))
- (search-path-specification
- (variable "CROSS_LIBRARY_PATH")
- (files '("arm-zephyr-eabi/lib")))))
- (home-page "https://zephyrproject.org")
- (synopsis "GCC for zephyr RTOS"))))</code></pre><p>This GCC can be built like so.</p><pre><code>guix build -L guix-zephyr gcc-cross-sans-libc-arm-zephyr-eabi
- /gnu/store/qmp8bzmwwimw0r6fh165hgfhkxkxilpj-gcc-cross-sans-libc-arm-zephyr-eabi-12.1.0-lib
- /gnu/store/38rli0rbn7ksmym3wq99cr4p2cjdz4a7-gcc-cross-sans-libc-arm-zephyr-eabi-12.1.0</code></pre><p>Great! We now have our stage-1 compiler.</p><h2>Newlib(-nano)</h2><p>The newlib package is quite straight forward (relatively).
- It is mostly adding in the relevent configuration flags and patching
- the files the <code>patch-shebangs</code> phase missed.</p><pre><code> (define-public zephyr-newlib
- (package
- (name "zephyr-newlib")
- (version "3.3")
- (source (origin
- (method git-fetch)
- (uri (git-reference
- (url "https://github.com/zephyrproject-rtos/newlib-cygwin")
- (commit "4e150303bcc1e44f4d90f3489a4417433980d5ff")))
- (sha256
- (base32 "08qwjpj5jhpc3p7a5mbl7n6z7rav5yqlydqanm6nny42qpa8kxij"))))
- (build-system gnu-build-system)
- (arguments
- `(#:out-of-source? #t
- #:configure-flags '("--target=arm-zephyr-eabi"
- "--enable-newlib-io-long-long"
- "--enable-newlib-io-float"
- "--enable-newlib-io-c99-formats"
- "--enable-newlib-retargetable-locking"
- "--enable-newlib-lite-exit"
- "--enable-newlib-multithread"
- "--enable-newlib-register-fini"
- "--enable-newlib-extra-sections"
- "--disable-newlib-wide-orient"
- "--disable-newlib-fseek-optimization"
- "--disable-newlib-supplied-syscalls"
- "--disable-newlib-target-optspace"
- "--disable-nls")
- #:phases
- (modify-phases %standard-phases
- (add-after 'unpack 'fix-references-to-/bin/sh
- (lambda _
- (substitute* '("libgloss/arm/cpu-init/Makefile.in"
- "libgloss/arm/Makefile.in"
- "libgloss/libnosys/Makefile.in"
- "libgloss/Makefile.in")
- (("/bin/sh") (which "sh")))
- #t)))))
- (native-inputs
- `(("xbinutils" ,zephyr-binutils)
- ("xgcc" ,gcc-arm-zephyr-eabi-12)
- ("texinfo" ,texinfo)))
- (home-page "https://www.sourceware.org/newlib/")
- (synopsis "C library for use on embedded systems")
- (description "Newlib is a C library intended for use on embedded
- systems. It is a conglomeration of several library parts that are easily
- usable on embedded products.")
- (license (license:non-copyleft
- "https://www.sourceware.org/newlib/COPYING.NEWLIB"))))</code></pre><p>And the build.</p><pre><code>guix build -L guix-zephyr zephyr-newlib
- /gnu/store/4lx37gga1jv3ckykrxsfgwy9slaamln4-zephyr-newlib-3.3</code></pre><h2>Complete toolchain</h2><p>Note that the toolchain is <em>Mostly</em> complete. libstdc++ does not build because
- `arm-zephyr-eabi` is not `arm-none-eabi` so a dynamic link check is
- performed/failed. I cannot figure out how crosstool-ng handles this.</p><p>Anyway, now that we’ve got the individual tools it’s time to create
- our complete toolchain. For this we need to do some package transformations.
- Because these transformations are must be done for every combination of
- binutils/gcc/newlib, it is best to create a function which we can reuse for
- every version of the SDK.</p><pre><code>(define (arm-zephyr-eabi-toolchain xgcc newlib version)
- "Produce a cross-compiler zephyr toolchain package with the compiler XGCC and the C
- library variant NEWLIB."
- (let ((newlib-with-xgcc (package (inherit newlib)
- (native-inputs
- (alist-replace "xgcc" (list xgcc)
- (package-native-inputs newlib))))))
- (package
- (name (string-append "arm-zephyr-eabi"
- (if (string=? (package-name newlib-with-xgcc)
- "newlib-nano")
- "-nano" "")
- "-toolchain"))
- (version version)
- (source #f)
- (build-system trivial-build-system)
- (arguments
- '(#:modules ((guix build union)
- (guix build utils))
- #:builder
- (begin
- (use-modules (ice-9 match)
- (guix build union)
- (guix build utils))
- (let ((out (assoc-ref %outputs "out")))
- (mkdir-p out)
- (match %build-inputs
- (((names . directories) ...)
- (union-build (string-append out "/arm-zephyr-eabi")
- directories)
- #t))))))
- (inputs
- `(("binutils" ,zephyr-binutils)
- ("gcc" ,xgcc)
- ("newlib" ,newlib-with-xgcc)))
- (synopsis "Complete GCC tool chain for ARM zephyrRTOS development")
- (description "This package provides a complete GCC tool chain for ARM
- bare metal development with zephyr rtos. This includes the GCC arm-zephyr-eabi cross compiler
- and newlib (or newlib-nano) as the C library. The supported programming
- language is C.")
- (home-page (package-home-page xgcc))
- (license (package-license xgcc)))))</code></pre><p>This function creates a special package which consists of the toolchain in a special directory hierarchy, i.e <code>arm-zephyr-eabi/</code>.
- Our complete toolchain definition looks like this.</p><pre><code>(define-public arm-zephyr-eabi-toolchain-0.15.0
- (arm-zephyr-eabi-toolchain
- gcc-arm-zephyr-eabi-12
- zephyr-newlib
- "0.15.0"))</code></pre><p>To build:</p><pre><code>guix build -L guix-zephyr arm-zephyr-eabi-toolchain
- /gnu/store/9jnanr27v6na5qq3dlgljraysn8r1sad-arm-zephyr-eabi-toolchain-0.15.0</code></pre><h1>Integrating with Zephyr Build System</h1><p>Zephyr uses CMake as it’s build system. It contains numerous CMake files in both the so-called <code>ZEPHYR_BASE</code>,
- the zephyr source code repository, as well as a handful in the SDK which help select the correct toolchain
- for a given board.</p><p>There are standard locations the build system will look for the SDK. We are not
- using any of them. Our SDK lives in the store, immutable forever. According to
- <a href="https://docs.zephyrproject.org/latest/develop/west/without-west.html">this
- webpage</a>,
- the variable <code>ZEPHYR_SDK_INSTALL_DIR</code> needs to point to our custom spot.</p><p>We also need to grab the cmake files from the <a href="https://github.com/zephyrproject-rtos/sdk-ng">repository</a> and create a file <code>sdk_version</code> which
- contains the version string <code>ZEPHYR_BASE</code> uses to find a compatible SDK.</p><p>Along with the SDK proper we need to include a number of python packages required by the build system.</p><pre><code> (define-public zephyr-sdk
- (package
- (name "zephyr-sdk")
- (version "0.15.0")
- (home-page "https://zephyrproject.org")
- (source (origin (method git-fetch)
- (uri (git-reference
- (url "https://github.com/zephyrproject-rtos/sdk-ng")
- (commit "v0.15.0")))
- (file-name (git-file-name name version))
- (sha256 (base32 "04gsvh20y820dkv5lrwppbj7w3wdqvd8hcanm8hl4wi907lwlmwi"))))
- (build-system trivial-build-system)
- (arguments
- `(#:modules ((guix build union)
- (guix build utils))
- #:builder
- (begin
- (use-modules (guix build union)
- (ice-9 match)
- (guix build utils))
- (let* ((out (assoc-ref %outputs "out"))
- (cmake-scripts (string-append (assoc-ref %build-inputs "source")
- "/cmake"))
- (sdk-out (string-append out "/zephyr-sdk-0.15.0")))
- (mkdir-p out)
- (match (assoc-remove! %build-inputs "source")
- (((names . directories) ...)
- (union-build sdk-out directories)))
- (copy-recursively cmake-scripts
- (string-append sdk-out "/cmake"))
- (with-directory-excursion sdk-out
- (call-with-output-file "sdk_version"
- (lambda (p)
- (format p "0.15.0")))
- #t)))))
- (propagated-inputs
- (list
- arm-zephyr-eabi-toolchain-0.15.0
- zephyr-binutils
- dtc))
- (native-search-paths
- (list (search-path-specification
- (variable "ZEPHYR_SDK_INSTALL_DIR")
- (files '("")))))
- (synopsis "SDK for zephyrRTOS")
- (description "zephyr-sdk contains bundles a complete gcc toolchain as well
- as host tools like dtc, openocd, qemu, and required python packages.")
- (license license:apsl2)))</code></pre><h2>Testing</h2><p>In order to test we will need an environment with the SDK installed.
- We can take advantage of <code>guix shell</code> to avoid installing test packages into
- our home environment. This way, if it causes problems, we can just exit the shell
- and try again.</p><pre><code>guix shell -L guix-zephyr zephyr-sdk cmake ninja git</code></pre><p><code>ZEPHYR_BASE</code> can be cloned into a temporary workspace to test our toolchain
- functionality (For now. Eventually we will need to create a package for
- <code>zephyr-base</code> that our guix zephyr-build-system can use).</p><pre><code>mkdir /tmp/zephyr-project
- cd /tmp/zephyr-project
- git clone https://github.com/zephyrproject-rtos/zephyr
- export ZEPHYR_BASE=/tmp/zephyr-project/zephyr</code></pre><p>In order to build for the test board (k64f in this case) we need to get a hold
- of the vendor Hardware Abstraction Layers and CMSIS (These will also need to
- become guix packages to allow the build system to compose modules).</p><pre><code>git clone https://github.com/zephyrproject-rtos/hal_nxp &&
- git clone https://github.com/zephyrproject-rtos/cmsis</code></pre><p>To inform the build system about this module we pass it in with <code>-DZEPHYR_MODULES=</code> which is
- a semicolon separated list of paths containing a module.yml file.</p><p>To build the hello world sample we use the following incantation.</p><pre><code>cmake -Bbuild $ZEPHYR_BASE/samples/hello_world \
- -GNinja \
- -DBOARD=frdm_k64f \
- -DBUILD_VERSION=3.1.0 \
- -DZEPHYR_MODULES="/tmp/zephyr-project/hal_nxp;/tmp/zephyr-project/cmsis" \
- && ninja -Cbuild</code></pre><p>If everything is set up correctly we will end up with a <code>./build</code>
- directory with all our build artifacts. The SDK is installed correctly!</p></div></article></section></main><footer><p>© 2020 Joshua Branson. The text on this site is free culture under the Creative Commons Attribution Share-Alike 4.0 International license.</p><p>This website is build with Haunt, a static site generator written in Guile Scheme. Source code is <a href="https://notabug.org/jbranso/gnucode.me">available.</a></p><p>The color theme of this website is based off of the famous <a href="#3f3f3f" target="_blank">zenburn</a> theme.</p></footer></body>
|
