title: cbmk maintenance manual x-toc-enable: true ...

NOTE: Canoeboot standardises on flashprog now, as of 3 May 2024, which is a fork of flashrom.

In addition to this manual, you should also refer to porting.md and testing.md.

Please also read about the cbmk coding style and design.

Automated coreboot build system

This document describes the entire Canoeboot build system, its design philosophy and how it's used to prepare Canoeboot releases; it is provided as a reference for Canoeboot development, pertaining to the current development branch of Canoeboot's build system (called cbmk).

The homepage of Canoeboot says that Canoeboot is a coreboot distro, providing the necessary integration of coreboot, payloads and utilities so as to provide releases, much like Linux distros do for your operating system, but here we are concerned about the boot firmware instead. Canoeboot is to coreboot, what Debian is to Linux. It provides easier, more automated configuration and installation.

The build system, cbmk, is that coreboot distro, at its very core. You can basically think of it as a package manager; it is even a source-based package manager. If you simply want to build ROM images, refer instead to the basic build instructions.

This build system, cbmk, is completely automated in every way. It is designed to take care of itself; so long as build dependencies are installed, it will check itself when running any command; if another command had to be executed first, it will do so automatically. Therefore, you can run any part of cbmk on its own, and the entire design is modular.

Best practises for learning cbmk

The follow sections will cover subdirectories, within cbmk. Contrary to what some may otherwise assume, it's best to learn about everything except scripts or code within Canoeboot, first. No, you should first learn about config files used in the Canoeboot build system, and then learn about the logic. By doing it in this order, you will have greater context later when reading about those scripts. Learning about each upstream project (such as coreboot) will also be useful; check documentation provided by each project.

After learning about configuration, you will then read about files and directories generated by the build system; only then will this document describe each script or program that forms part of the build system. In other words, this document adopts a top-down approach to education, rather than bottom-up; most documents take the latter approach, in other projects, but most people naturally want to learn how a specific thing works first, hence the approach taken here.

Don't be deceived by simplicity

Canoeboot's build system is powerful, and highly configurable, yet deceptively simple at the same time. Remember this rule, a rule that applies to all software projects: code equals bugs, so smaller codebases will yield fewer bugs. Canoeboot is regularly audited.

Many people will be shocked by how small Canoeboot is, at its core. You will be surprised by just how much can be done with so little. Continue reading!

Environmental variables

XBMK_THREADS

For example:

export XBMK_THREADS=2

This would build on two threads, when running cbmk. It defaults to 1.

Previous revisions of cbmk used nproc by default, but this was set to 1 instead, because nproc is not available on every operating system.

XBMK_RELEASE

If set to y, it signals to script/roms that a release is being built, and it will honour release="n" in target.cfg files. You could also set this yourself, when doing regular builds, if you wanted to test how ./build roms behaves running it in release mode. Do this if you want to:

export XBMK_RELEASE=y

Projects/files downloaded/generated by cbmk

The following sections will describe files and directories that are not included in cbmk.git, but are created by running various cbmk commands; many of these will also be provided, pre-generated, under release archives.

Some of these are downloaded by Canoeboot's build system, automatically, while others are created during the build process based on these downloaded programs.

bin/

This directory is created when running any of the following commands, with the right arguments:

./mk -b coreboot ARGUMENTS_HERE
./mk -b stm32-vserprog
./mk -b pico-serprog

Simply speaking, bin/ shall contain finished ROM images or firmware, that can then be installed (flashed) to the target device.

The files under bin/ are provided in regular Canoeboot releases.

These are the ROM images that you should flash.

Older versions of lbmk build coreboot images separately under elf/, but without payloads, using elf/ as a build cache, then inserting payloads into copies of these images in files under bin/. However, modern lbmk now only puts coreboot images in bin/, with payloads included.

If you still have elf/ coreboot images in your lbmk tree, please do not use them (and you may aswell delete them).

cache/

Certain files are cached here automatically, by lbmk. The user need not touch these files.

cache/app

When vendor updates are extracted, they go here, which is then processed to find individual files for use in coreboot images (e.g. KBC1126 EC firmware).

This directory is constantly over-written, so it's essentially another temporary directory used by the build system.

cache/file/

Files that are downloaded are hashed, and the cached version of the file is stored there, named as the SHA512 checksum. This is used for vendor file downloads, and subfile downloads.

A subfile is like a Git submodule, but it's a file (just a humble file), downloaded via curl/wget. The build system does not run git submodule update commands when handling Git repositories anymore, instead processing submodules manually; it supports both repositories and files relative to the directory locations for those repositories, but subfiles are not downloaded to the cached git repository, only the work directory used for building in lbmk.

cache/hash

When lbmk is handling any project, it sorts a list of files under config/ including config/project (or config/project/TREE) and config/data/project.

SHA512 checksums are calculated from these files, in the sorted order, and written in that order, to a file. That file is then checksummed, and this hash is stored in cache/hash for that project.

If the currently stored hash differs from what's calculated, it means that the project has changed, and the source directories plus builds are deleted. The project source is then re-prepared and re-build.

cache/repo

Git repositories are cached here. This avoids wasting bandwidth, when downloading multiple repositories. Git submodules are also cached here!

elf/

DO NOT flash coreboot ROM images contained under elf/. Please use ROM images under bin/ instead! - In modern lbmk, only the ones under bin/ are ever created anyway.

Compiled binaries (compiled by cbmk) go here, but they are not the final binaries; coreboot ROM images are compiled without payloads, then cached here under elf/coreboot as one example

GRUB and SeaBIOS which go under elf/grub and elf/seabios respectively - elf/u-boot is another example. A given project can include a build.list file at config/data/PROJECT/build.list, which would contain a list of file paths relative to the source directory; these files would be copied, after a build operation, to elf/PROJECT for single-tree projects, or elf/PROJECT/TREE for multi-tree projects.

It is technically possible to re-use these files elsewhere. For example, you may wish to only compile GRUB with lbmk, and then use the grub.elf file from cbmk in your own custom coreboot ROM (that you didn't build with lbmk). However, this use is not officially supported by the Canoeboot project; these files are simply used by the Canoeboot build system.

Some utilities are also provided compiled here, when building. For example: elf/flashprog/flashprog. This is because lbmk tries to provide out-of-source builds whenever feasible.

This is only used by the build system, but these images are not provided in releases (only the images under bin/ are provided).

As of Canoeboot 20240612, the elf/ directory must be used by default for all builds, in an effort to make exclusive use of out-of-source builds. As such, the cbutils directory is no longer used.

release/

The script at build create tarballs in here, which constitute regular Canoeboot releases. It is meticulously maintained, as per current cbmk behaviour, and executed so as to provide Canoeboot release archives.

This provides source tarballs, and ROM images.

You can create release archives by doing:

./mk release

By default, this creates a release under release/, but you can change the directory, for example:

./mk release -d path

You can also specify that only a source archive be created, like so:

./mk release -m src

Or with a custom directory:

./mk release -d path -m src

The build system expects there to be a git tag, so make sure there is one. This is used to create the version number for a given release.

src/

Third-party source trees are downloaded into this directory, by cbmk.

src/coreboot/

Please also visit: https://coreboot.org/

Coreboot is the main boot firmware, providing hardware initialisation. Canoeboot makes extensive use of coreboot, on supported mainboards.

Coreboot trees go here. Canoeboot's build system does not simply use one tree, or multiple branches in the same tree; entirely separate directories are created, for each revision of coreboot used, each able to have its own patches. These can then be re-use appropriately, per mainboard. For example:

src/coreboot/default is used by most mainboards.
src/coreboot/cros is used by cros devices.

This may be less efficient on disk usage, but it simplifies the logic greatly. Coreboot also uses its own toolchain called crossgcc, and crossgcc is in fact compiled per tree in Canoeboot.

src/flashprog/

Please also visit: https://flashprog.org/

Although currently unused by any part of cbmk, we provide flashprog for the convenience of users, and this is copied to release archives. Flashrom is the program that you will use to read, erase and write the flash, containing coreboot firmware.

src/grub/TREE

Please also visit: https://www.gnu.org/software/grub/

The GNU GRUB bootloader, a reference multiboot implementation with its own small kernel/OS and drivers (e.g. file systems, cryptography). This is the default recommended coreboot payload on x86-based Canoeboot systems. GRUB will load and execute your Linux kernel, which then runs on the bare metal.

The utilities for GRUB are compiled here, and used from here; specifically, the grub-mkstandalone utility is executed from here to create the final GRUB image under elf/grub/.

NOTE: This is only provided for x86 machines, in Canoeboot. For ARM, we ship U-Boot instead. Since Canoeboot 20240612, the GRUB builds are multi-tree, much like, say, coreboot or SeaBIOS.

As of August 2024, the following GRUB source trees can be downloaded:

src/grub/default
src/grub/xhci
src/grub/nvme

Simplify specify the tree. For example:

./mk -b grub xhci

The xhci tree contains patches for both NVMe SSD support, and xHCI. The nvme tree contains NVMe SSD support but not xHCI support. The default tree contains no NVMe or xHCI support. All trees otherwise have the same fixes on top of upstream GRUB, e.g. fix for Dell Latitude keyboard controllers.

src/memtest86plus/

Please also visit: https://www.memtest.org/

This is provided inside ROM images, as a payload executed from main GRUB or SeaBIOS payload. It checks for corrupted memory.

src/seabios/

Please also visit: https://www.seabios.org/SeaBIOS

This is the PC BIOS implementation used by Canoeboot, on x86 machines (not all of them). A BIOS/UEFI implementation is not required, because Linux and BSD kernels can execute on bare metal, but it can nonetheless still be useful; in particular, the BSD bootloaders can be executed from SeaBIOS.

This is provided as a coreboot payload, either as first payload or it can be executed from GRUB (if GRUB is the main payload, on a given target).

src/u-boot/

Please also visit: https://www.denx.de/project/u-boot/

This is a bootloader provided on ARM chromebooks, within Canoeboot. It also provides UEFI. Information about that can be found on these resources:

This is currently the only payload on ARM systems, within Canoeboot.

src/pico-serprog

Used by cbmk, to build firmware for serprog-based SPI flashers with RP2040 SoC. Alongside this, util-fw/rp2040/pico-sdk is imported which is required for building it.

Please visit these pages:

src/stm32-vserprog

Used by cbmk, to build firmware for serprog-based SPI flashers with STM32 MCU. Alongside this, libopencm3 is imported which is required for building it.

These serprog programmers are quite desirable, owing to their low cost and ease of use. You can learn more on the SPI flashing guide.

Before moving onto configurations, we will now cover utilities provided by Canoeboot itself (included within cbmk, rather than being downloaded like the third party projects listed above):

tmp/

The TMPDIR environmental variable is set by cbmk, to a location under /tmp, but some users may have /tmp mounted as a tmpfs (file system in RAM), and may not have much RAM.

Where large files (or a large number of files) are handled by cbmk on a temporary basis, this tmp/ directory is created and then used.

util/

If a codebase is not frequently used by Canoeboot, is actively developed (making it not viable to maintain in Canoeboot) or the codebase is very large, we would import that as a third party module in cbmk - this rule exists for all projects, where the intention is that cbmk.git itself should be small and efficient.

Where appropriate, and where the code is small enough, or it is otherwise deemed desirable, cbmk.git provides a few utilities as part of itself, namely:

util/dell-flash-unlock/

This program, written by Nicholas Chin, unlocks the boot flash on Dell Latitude E6400; it permits internal flashing, from factory firmware to Canoeboot, so that the user need not disassemble and flash externally.

It also supports several other Dell laptops, with similar ECs. Check the README file included in this directory, for more information.

util/nvmutil/

The nvmutil software allows you to set the MAC address on Intel GbE NVM files. It also allows you to set random MAC addresses, in addition to arbitrary ones.

This directory contains the source code for nvmutil, which you can read about here:

nvmutil manual

util/spkmodem_recv/

FSF has original copyright on this; it was imported from coreboot, who in turn imported it from GRUB with very little modification. Therefore, this code is canonically based on what is provided in GNU GRUB.

This is a receiving client for spkmodem, which is a method of providing serial consoles via pulses on the PC speaker. The spkmodem_recv client will decode these pulses. Coreboot has a driver for generating these pulses, as does GRUB; this client code was imported from GRUB, and has in fact been provided by every Canoeboot release since the start of the project (look inside the GRUB or coreboot source code and you'll find it).

However, the original code from GRUB was of quite poor quality and this code is often used. For fun, it was decided that this utility would be imported directly into cbmk.git, and thoroughly cleaned. The cbmk version has been more or less re-written, using the original logic as a base; variables are more clearly named. A top-down, OpenBSD-inspired coding style is used, replacing the GNU coding style implemented in the original code. The [OpenBSD coding style][https://man.openbsd.org/style.9] is much easier to read.

This code has been modified to make use of the pledge() system call, when used on OpenBSD; the original version from GRUB did not do this. Other improvemnts include:

Superior error handling (the program actually exits with non-zero status now, under fault conditions, whereas the original code did not handle errors).
Debug mode is now handled via getopt() by passing the -d flag at run time, whereas the original code only enabled it if a DEBUG build-time flag was used.
The code has been translated into English (e.g. references to "trames" in the code, now say "frames" in the Canoeboot version).
Certain magic numbers, and certain equations in code, are now labelled as either variables or as #define values, thus increasing code legibility.

Now in the next sections, you will learn about configuration files provided by cbmk:

config/

This directory contains configuration files, used by the Canoeboot build system. These next sections will cover specific configuration files.

config/PROJECT*/nuke.list

The script include/git.sh handles deletion of certain files, for downloaded projects, based on a nuke.list file that can (for single-tree projects) be included at config/PROJECT/nuke.list or (multi-tree project) at config/PROJECT/TREE/nuke.list (entries are relative links from the root directory of the given source tree e.g. src/coreboot/default/).

So, if src/coreboot/default/ contained foo/bar.txt, you could add to the nuke.list file as follows:

foo/bar.txt

Ditto src/flashprog/, if you wanted to delete a file from in there, as one other example. Deletions occur when the source tree is created.

config/coreboot

config/coreboot/BOARDNAME/

Each target name (e.g. x200_8mb) has its own directory under here. Targets that do not define defconfigs also exist here; for example, the default directory defines a coreboot revision and patches.

Targets under config/coreboot can specify tree=TREE where TREE could, for example, be default. In other words, they can refer to other trees.

The coreboot downloads are based on scanning of these directories, and ROM images are also built based on them.

config/coreboot/BOARDNAME/patches/

For any given coreboot tree, patches with the patch file extension are placed here, alphanumerically in the order that they should be applied.

These patches are then so applied, when cbmk downloads the given source tree.

config/coreboot/BOARDNAME/target.cfg

This file can contain several configuration lines, each being a string, such as:

tree="default" (example entry)
rev="ad983eeec76ecdb2aff4fb47baeee95ade012225" (example entry)
xarch="i386-elf" (example entry)
payload_grub="y" (example entry)
payload_seabios="y" (example entry)
payload_memtest="y" (example entry)
payload_uboot="y" (example entry)
grub_scan_disk="ata"
uboot_config=default (specify which U-Boot tree to use)
release="n" (example entry)
xtree="default" (example entry)
tree_depend="default" (example entry)
grubtree="nvme" (example entry)

The tree value refers to config/coreboot/TREE; in other words, a given target could specify a name other than its own as the tree; it would then re-use code from that tree, rather than providing its own.

The rev entry defines which coreboot revision to use, from the coreboot Git repository. At present, cbmk only supports use of the official repository from the upstream coreboot project.

The xarch entry specifies which CPU architecture is to be used: currently recognized entries are i386-elf, arm-eabi and aarch64-elf. This is the target architecture for building GCC/toolchain from coreboot crossgcc, hence xarch.

The payload_grub entry specifies whether or not GRUB is to be included in ROM images.

The payload_seabios entry specifies whether or not SeaBIOS is to be included in ROM images. If GRUB is also enabled, standalone SeaBIOS images will be created alongside SeaGRUB images. SeaGRUB is where SeaBIOS automatically loads GRUB, via bootorder inserted into CBFS.

The payload_memtest entry specifies whether or not MemTest86+ is to be included in ROM images; it will only be included in ROM images for text mode startup, on x86 machines.

The payload_uboot entry specifies whether or not U-Boot is to be included in ROM images.

The uboot_config option specifies which U-Boot board configuration file variant should be used. It currently doesn't make sense for this to be anything other than default, which is the default if the option is missing.

The grub_scan_disk option specifies can be ahci, ata or both, and it determines which types of disks are to be scanned, when the grub.cfg file in GRUB payloads tries to automatically find other grub.cfg files supplied by your Linux distribution. On some machines, setting it to ata or ahci can improve boot speed by reducing delays; for example, trying to scan ata0 on a ThinkPad X60 with the optical drive may cause GRUB to hang, so on that machine it is advisable to set this option to ahci (becuse the default HDD slot is AHCI).

The release variable can be set to n, which makes the ./mk release call skip that target, when creating release images. For example, a given board may not be stable and you don't want images for it to be included in the release.

The xtree option specifies that a given tree with use a specific coreboot tree for compiling crossgcc. This can be used to skip building gcc if OK on a given board; two trees may use the same crossgcc as each other.

The tree_depend option means that a given tree needs another tree, defined by this variable, to also be present.

The grubtree option specifies which GRUB tree to use. If unset, it defers to the default GRUB tree.

config/coreboot/BOARDNAME/config/

Files in this directory are coreboot configuration files.

Configuration file names can be as follows:

libgfxinit_corebootfb
libgfxinit_txtmode
vgarom_vesafb
vgarom_txtmode
normal

Information pertaining to this can be found on the installation manual

In cbmk, a board-specific directory under config/coreboot/ should never specify a coreboot revision. Rather, a directory without coreboot configs should be created, specifying a coreboot revision. For example, the directory config/coreboot/default/ specifies a coreboot revision. In the board-specific directory, your board.cfg could then specify cbtree="default" but without specifying a coreboot revision (this is specified by config/coreboot/default/board.cfg).

When you create a coreboot configuration, you should set the payload to none because cbmk itself will assume that is the case, and insert payloads itself.

Configurations with libgfxinit will use coreboot's native graphics init code if available on that board. If the file name has txtmode in it, coreboot will be configured to start in text mode, when setting up the display. If the file name has corebootfb in it, coreboot will be configured to set up a high resolution frame buffer, when initializing the display.

NOTE: If the configuration file is libgfxinit_txtmode, the SeaBIOS payload can still run external VGA option ROMs on graphics cards, and this is the recommended setup (SeaBIOS in text mode) if you have a board with both onboard and an add-on graphics card (e.g. PCI express slot) installed.

Configuration files with vgarom in the name have coreboot itself configured to run VGA option ROMs (and perhaps other option ROMs). This setup is not strictly recommended for SeaBIOS, and it is recommended that you only run GRUB in this setup. As such, if you wish for a board to have coreboot initialize the VGA ROM (on an add-on graphics card, as opposed to onboard chipset), you should have a separate directory just for that, under config/coreboot/; another directory for that board will have configs with libgfxinit. HOWEVER:

It is supported in cbmk to have SeaBIOS used, on either setup. In the directory config/seabios/ there are SeaBIOS configs for both; the vgarom one sets VGA hardware type to none while the libgfxinit one sets it to coreboot linear framebuffer. However, if you use SeaBIOS on a setup with coreboot also doing option ROM initialization, such initialization is being performed twice. As such, if you want to use an add-on graphics card in SeaBIOS, but the board has libgfxinit, it is recommended that you do it from a libgfxinit ROM.

HOWEVER: there's no hard and fast rule. For example, you could make a vgarom configuration, on a board in cbmk, but in its coreboot configuration, don't enable native init or oproms, and do SeaBIOS-only on that board.

On vgarom setups, coreboot can be configured to start with a high resolution VESA frame buffer (NOT to be confused with the coreboot frame buffer), or just normal text mode. Text mode startup is always recommended, and in that setup, GRUB (including coreboot GRUB, but also PC GRUB) can use VGA modes.

The name libgfxinit is simply what ./mk -b coreboot uses, but it may be that a board uses the old-school native video init code written in C. On some platforms, coreboot implemented a 3rd party library called libgfxinit, which is written in Ada and handles video initialization. In this setup, coreboot itself should never be configured to run any option ROMs, whether you start in text mode or with the coreboot framebuffer initialization.

The normal config type is for desktop boards that lack onboard graphics chipsets, where you would always use an add-on graphics card (or no graphics card, which would be perfectly OK on servers).

Even if your board doesn't actually use libgfxinit, the config for it should still be named as such. From a user's perspective, it really makes no difference.

config/dependencies/

Files here are so named, and called like so: e.g. the debian file would be referenced when running:

./mk dependencies debian

These files define a list of packages, and the correct package manager command to use on a given distro. This can be used to install build dependencies, which are required for compiling Canoeboot from source code.

config/git/

Configuration related to third-party Git repositories, that Canoeboot makes use of.

These file define third party codebases, with repository links, revision IDs, and dependencies (referring to other modules defined in this file).

Almost every third party codebase that cbmk downloads is based on the handling of this file. Some of the codebases defined here will also have a directory of their own; for example, config/grub/ exists.

Multiple files exist here, and they are concatenated in a temporary file by cbmk, which is then scanned to find information about projects.

config/data/PROJECT/mkhelper.cfg

These mkhelper.cfg files define common configuration that can be supplied for any single- or multi-tree project. Arguments available are as follows:

makeargs: This defines what arguments to append when running the main make command on a given project. For example, this is used on coreboot to tell coreboot's build system that the submodules have been updated (to avoid downloading any that we didn't manually specify).
build_depend: Just before running the main make command on a given project, this specifies other projects to build. It also works with multi tree projects. Example: seabios/default grub/xhci memtest86plus
premake: This defines a function to be called before running make, on a given project; the mkhelper file itself can also import any given file to provide that function.
mkhelper (variable name): Defines a function to be called just after running make, on a given project.
postmake: This is run after mkhelper, and can be used for additional functions. For example, it's used on coreboot to call mkcoreboottar which will create tarballs of ROM images if XBMK_RELEASE is enabled.

You can define anything else here, for use by a given project. More specifically, anything you put in mkhelper files will be imported as part of a normal shell script during operation of lbmk, to complement core functionality across all the various projects.

The mkhelper file is a global configuration for the project. Individual projects can complement what is set in mkhelper, via target.cfg files for each project, project tree or target on a given multi-tree project.

The mkhelper functionality (and postmake/premake) was originally implemented so that lots of special configuration could be done per project, without a lot of code repetition. This is a unique design of lbmk, different from many other coreboot-distro build systems.

The mkhelper functionality is an essential component that makes lbmk work the way it does; for example, the trees script builds coreboot images without payloads, and functions to add payloads are handled by mkhelper-type functions. This design allows almost all functionality to be centralised, where the mkhelper functions only provide functionality that differs from core functionality.

In the simplest of terms, you may regard mkhelpers as plugins, of a sort. They simply extend the core functionality of the build system, in a way that can differ flexibly between projects.

GRUB config

config/data/grub/background

Splash screen images applied duing startup when using the GRUB payload.

config/data/grub/background/background1024x768.png

Used on ThinkPad X60 and T60.

config/data/grub/background/background1280x800.png

Used on all other machines, besides X60 and T60 thinkpads.

NOTE: the grub_background option can be set under target.cfg in the relevant coreboot directory, under config/coreboot/; for example, config/coreboot/x60/target.cfg specifies this:

grub_background="background1024x768.png"

config/data/grub/background/COPYING

Licensing info for GRUB bootsplash images.

config/grub/TREE/config/

GRUB configuration files.

config/grub/config/AUTHORS

Author info for GRUB configuration files.

config/grub/config/COPYING

Licensing info for GRUB configuration files.

config/grub/TREE/config/payload

This is a configuration file. It is used to program GRUB's shell.

This is inserted (as grub.cfg) into the GRUB memdisk, in the ROM image. It contains a lot of logic in it, for booting various system configurations, when the GRUB payload is in use.

It can be overridden by inserting grub.cfg into coreboot's main CBFS root.

A grubtest.cfg can be inserted into CBFS, but it will not override the default grub.cfg (either in CBFS or on memdisk); however, the one in memdisk will provide a menuentry for switching to this, if available.

config/data/grub/memdisk.cfg

This GRUB configuration checks whether grub.cfg exists in CBFS and switches to that first (not provided by default) or, if one is not available in CBFS, it will load the grub.cfg stored inside GRUB memdisk.

The GRUB memdisk is a file system within grub.elf, itself stored within the coreboot file system named CBFS, which is part of the coreboot ROM image on every coreboot target.

config/data/grub/keymap/

Keymap files used by GRUB. They can alter the character set corresponding to inputted scancodes.

config/data/grub/keymap/*.gkb

The keymap files themselves. These are inserted into the GRUB memdisk, and the grub.cfg file can specify which one is to be used.

These files are binary-encoded, defining which characters correspond to which scancodes. It is handled by grub-core/commands/keylayouts.c in the GRUB source code.

config/data/grub/module/TREE

This defines which modules are inserted into grub.elf. These modules can be anything from file systems, small applications/utilities, launchers (e.g. the linux command will execute a Linux kernel), you name it.

Canoeboot defines only a very conservative set of modules here, so as to reduce the amount of space used in the main boot flash. (GRUB payloads are also compressed when they are inserted into coreboot images)

This list is used by cbmk when it runs grub-mkstandalone, which is the utility from GRUB that generates grub.elf files (to be compressed inside CBFS and then executed as a coreboot payload).

config/grub/TREE/patches/

For a given GRUB revision, patches with the patch file extension are placed here, alphanumerically in the order that they should be applied. For example, Canoeboot provides argon2 key derivation support out of tree, allowing LUKS2 partitions to be decrypted by GRUB.

These patches are then so applied, when cbmk downloads the given source tree.

config/ifd/*

Intel Flash Descriptors and GbE NVM images, which are binary-encoded configuration files. These files are referenced in coreboot defconfigs, used by cbmk to build coreboot ROM images.

config/seabios/

config/data/seabios/build.list

When a given SeaBIOS tree is compiled, for a given target, this file defines which files to copy from the seabios/ directory, which are then copied to a location under elf/seabios.

config/seabios/default/

Currently the only tree in use, this defines what SeaBIOS revision is to be used, when the SeaBIOS payload is enabled on a given coreboot target.

config/seabios/default/config/

Configuration files go in here.

config/seabios/default/config/libgfxinit

Configuration file for when native video initialisation is available in coreboot.

config/seabios/default/config/normal

Configuration file for when native video initialisation is unavailable in coreboot, and VGA ROM initialisation is also not provided by coreboot (in this configuration, the usual setup will be that SeaBIOS finds and executes them, instead of coreboot).

config/seabios/default/config/vgarom

Configuration file for when native video initialisation is unavailable in coreboot, and VGA ROM initialisation is provided by coreboot; in this setup, SeaBIOS should not execute VGA ROMs.

config/seabios/default/target.cfg

Similar concept to target.cfg files provided by coreboot. This specifies which SeaBIOS revision (from Git) is to be used, when compiling SeaBIOS images.

config/u-boot/

This directory contains configuration, patches and so on, for each mainboard that can use U-Boot as a payload in the cbmk build system. U-Boot doesn't yet have reliable generic configurations that can work across all coreboot boards (per-architecture), so these are used to build it per-board.

config/data/u-boot/build.list

When a given U-Boot tree is compiled, for a given target, this file defines which files to copy from the U-Boot source build, which are then copied to a location under elf/u-boot/.

config/u-boot/TREENAME/

Each TREENAME directory defines configuration for a corresponding mainboard. It doesn't actually have to be for a board; it can also be used to just define a U-Boot revision, with patches and so on. To enable use as a payload in ROM images, this must have the same name as its config/coreboot/TREENAME/ counterpart.

config/u-boot/TREENAME/patches/

For any given U-Boot tree, patches with the patch file extension are placed here, alphanumerically in the order that they should be applied.

These patches are then so applied, when cbmk downloads the given source tree.

config/u-boot/TREENAME/target.cfg

This file can contain several configuration lines, each being a string, such as:

tree="default" (example entry)
rev="4debc57a3da6c3f4d3f89a637e99206f4cea0a96" (example entry)
arch="AArch64" (example entry)

These are similar in meaning to their coreboot counterparts.

The treeentry is actually a link, where its value is a directory name underconfig/u-boot. For example,tree="default"would refer toconfig/u-boot/defaultand the corresponding U-Boot source tree created (when running./mk u-boot, which makes use oftarget.cfg) would beu-boot/default/. In other words: atarget.cfgfile inconfig/u-boot/foomight refer toconfig/u-boot/barby specifyingtree="bar", and the created u-boot source tree would beu-boot/bar/`. ALSO:

FUN FACT: such references are infinitely checked until resolved. For example, foo can refer to bar and bar can refer to baz but if there is an infinite loop, this is detected and handled by cbmk. For example, if bar refers to foo which refers back to bar, this is not permitted and will throw an error in cbmk.

The rev entry defines which U-Boot revision to use, from the U-Boot Git repository. At present, cbmk only supports use of the official repository from the upstream U-Boot project.

The arch entry specifies which CPU architecture is to be used: currently recognized entries are x86_32, x86_64, ARMv7 and AArch64. Setting it to a non-native arch means that necessary crossgcc-arch will be compiled and be available when building roms, but not necessarily built or discovered when individual scripts are called manually.

config/u-boot/TREENAME/config/

Files in this directory are U-Boot configuration files. Configuration file names can be anything, but for now default is the only one used.

In cbmk, a board-specific directory under config/u-boot/ should never specify a U-Boot revision. Rather, a directory without U-Boot configs should be created, specifying a U-Boot revision. For example, the directory config/u-boot/default/ specifies a U-Boot revision. In the board-specific directory, your board.cfg could then specify ubtree="default" but without specifying a U-Boot revision (this is specified by config/u-boot/default/board.cfg).

Normally, the U-Boot build process results in the U-Boot executable and a device-tree file for the target board, which must further be packaged together to make things work. When you create a U-Boot configuration, you should enable CONFIG_REMAKE_ELF or CONFIG_OF_EMBED that handles this. The former option enables creation of a u-boot.elf that bundles them together after the build, and the latter option embeds it into the u-boot executable.

When making a U-Boot configuration, you should also pay special attention to the CONFIG_SYS_TEXT_BASE (CONFIG_TEXT_BASE in later versions), whose defaults may cause it to overlap coreboot, in which case it won't boot. Normally, the upstream coreboot build system checks for this when given CONFIG_PAYLOAD_ELF, but cbmk injects the payload itself and doesn't check for this yet.

Another interesting config option is CONFIG_POSITION_INDEPENDENT for ARM boards, which has been so far enabled in the ones cbmk supports, just to be safe.

config/submodule

In here you can find submodule configurations for projects. It works for both single- and multi-tree projects. Use the existing examples as reference.

Files, in each directory:

module.list lists paths (files and directories) for given modules, which can be files(via URL) or Git repositories, or both.
NAME/module.cfg

NAME is the file/directory name for the module, with everything up to the final forward slash removed. E.g. foo/bar/thing.zip would be thing.zip as NAME.

In module.cfg there can be either, file:

subfile="url"
subfile_bkup="url"
subhash="sha512sum for file"

or, git repository:

subrepo="url"
subrepo_bkup="url"
subhash="sha1 git commit id"

You must only use subfile or subrepo, not both, and there must be a backup URL. The build system intentionally avoids using Git's actual submodules feature, instead opting to download such repositories manually, because the official submodules feature doesn't have very good redundancy.

Additionally, a patches directory can be included alongside module.cfg, which can be used to patch the submodule (only supported for Git repositories because files are not extracted, only placed at their configured destination).

The destination path in module.list is relative to the location of the main Git repository under which it is placed.

config/data/PROJECT

Random configuration data provided on a per-project basis. Complements the config/PROJECT directory.

U-Boot build system

If you wish to know about U-Boot, refer here:\ https://u-boot.readthedocs.io/en/latest/

This and other documents from U-Boot shall help you to understand U-Boot.

You create a config, for config/u-boot/TREENAME/configs, by finding the corresponding board name in the upstream U-Boot configs directory, and running make BOARDNAME_defconfig and make menuconfig commands in the U-Boot build system. You should do this after running ./mk u-boot in cbmk.

You might want to consider basing your config on the upstream coreboot boards when possible, but such a board is not available upstream for ARM yet.

You can simply clone U-Boot upstream, add whatever patches you want, and then you can make your config. It will appear afterwards in a file named .config which is your config for inside config/u-boot/TREENAME/.

You can then use git format-patch -nX where X is however many patches you added to that U-Boot tree. You can put them in the patches directory under config/u-boot/BOARDNAME.

The base revision, upon which any custom patches you wrote are applied, shall be the rev entry.

Scripts exist in cbmk for automating the modification/updating of existing configs, but not for adding them. Adding them is to be done manually, based on the above guidance.

Config files in cbmk root directory

projectsite

Domain name linking to the project home page (e.g. canoeboot.org).

projectname

This is a text file, containing a single line that says canoeboot. This string is used by the build system, when naming releases alongside the version number.

version

Updated each time cbmk runs, based on either git describe or, on release archives, this file is static and never changes. It says what Canoeboot revision is currently in use (or was in use, if cbmk isn't running).

versiondate

Updated each time cbmk runs, based on either git describe or, on release archives, this file is static and never changes. It says the time of whichever Canoeboot revision is currently in use (time of commit).

At last, you will now learn about the scripts (exclusively written as posix shell scripts) that constitute the entire Canoeboot build system, cbmk:

Scripts in root directory of cbmk

build

This is the main script. Symlinks vendor and update also point to it, as does mk. Direct use of this script is considered deprecate, because in a future audit, it will be removed; the ./mk commands will be used, exclusively, so please use only ./mk as directed here and elsewhere.

Take any given file under script/ and you can do:

./build file # (THIS IS NOT A VALID COMMAND)

For example:

./mk -b coreboot
./mk

Special commands available (not provided by files under script/):

./mk release
./mk -d coreboot TARGET # also downloads vendor files

Information about ./mk release is written elsewhere on this page.

You can also know what build system revision you have by running:

./mk version

This script is the beating heart of Canoeboot. Break it and you break Canoeboot.

include/

This directory contains helper scripts, to be included by main scripts using the . command (called the source command in bash, but we rely upon posix sh only).

include/git.sh

These functions in here previously existed as independent scripts, but they were unified here, and they are used when you pass the -f argument to script/update/trees (e.g. ./mk -f coreboot).

These functions deal with git cloning, submodule updates, revision resets and the application of patch files via git am. Every git repository downloaded by cbmk is handled by the functions in this file.

include/mrc.sh

This was previously a separate script. The download logic was removed, and now the logic under include/vendor.sh is used for downloads. This file now only contains those functions used for extraction of MRC files from Google Chromebook images, currently only used for Haswell mainboards.

This is an include, used by include/vendor.sh, but it's here in this file because the vendor download script is GPLv3-only, while the MRC extract logic in this file is GPLv2-only (forked from coreboot ages ago). Thus, it is provided as an include to bypass license incompatibility. It has been heavily modified to use the same style of logic and general control flow used in the script at include/vendor.sh, and it is used from there.

c18402c8 (update docs/maintain/)

include/lib.sh

Several other parts of cbmk also use this file. It is added to as little as possible, and contains miscallaneous functions that don't belong anywhere else.

The functions here are mostly those that deal with configuration files; scanning them to set variables and so on.

This file also contains generic error handling, used by all cbmk scripts.

This also contains functions to verify the current Canoeboot version, and check whether Git is properly initialised on the host system. It also contains the setvars function, which provides a shorthand way of initialising many variables (combined with use of eval), which cbmk uses heavily.

This function also contains x_() which cbmk uses to execute commands and ensure that they cause an exit (with non-zero status) from cbmk, if they return an error state.

include/rom.sh

This builds coreboot ROM images. Specifically, this contains mkhelper functions. It also builds serprog images, and it could be used to provide functions for building other types of firmware.

Command: ./mk -b coreboot targetname

The targetname argument must be specified, chosen from this output:

./mk -b coreboot list

Pass several board names if you wish to build only for specific targets. For example:

./mk -b coreboot x60 x200_8mb

To build all targets, specify:

./mk -b coreboot

For x86 targets, these scripts build with the GRUB and/or SeaBIOS payloads inserted into the ROM images; secondary payloads like Memtest86+ are also handled and inserted here.

It heavily makes use of the target.cfg file, for a given board. This script will only operate on a single target, from a directory in config/coreboot/.

If grub_scan_disk is set, it sets that in the scan.cfg file that is to be inserted into a ROM image, when payload_grub is turned on.

It automatically detects if crossgcc is to be compiled, on a given coreboot tree (in cases where it has not yet been compiled), and compiles it for a target based on the arch entry in target.cfg.

It creates ROM images with GRUB, SeaBIOS, U-Boot, optionally with Memtest86+ also included, in various separate configurations in many different ROM images for user installation.

If no payload is defined in target.cfg, the build/roms script will exit with error status.

If SeaBIOS is to be used, on libgfxinit setups, SeaVGABIOS will also be inserted. This provides a minimal VGA compatibility layer on top of the coreboot framebuffer, but does not allow for switching the VGA mode. It is currently most useful for directly executing ISOLINUX/SYSLINUX bootloaders, and certain OS software (some Windows setups might work, poorly, depending on the board configuration, but don't hold your breath; it is far from complete).

If SeaBIOS is to be used, in vgarom setups or normal setups, SeaVGABIOS is not inserted and you rely on either coreboot and/or SeaBIOS to execute VGA option ROMs.

In all cases, this script automatically inserts several SeaBIOS runtime configurations, such as: etc/ps2-keyboard-spinup set to 3000 (PS/2 spinup wait time), etc/pci-optionrom-exec set to 2 (despite that already being the default anyway) to enable all option ROMs, unless vgarom setups are used, in which case the option is set to 0 (disabled) because coreboot is then expected to handle option ROMs, and SeaBIOS should not do it.

This script handles U-Boot separately, for ARM-based chromeos devices.

When the ROM is finished compiling, it will appear under a directory in bin/

This script is the beating heart of Canoeboot. Break it, and you break Canoeboot!

Serprog images:

Build firmware images for serprog-based SPI programmers, where they use an STM32 MCU. It also builds for RP2040-based programmers like Raspberry Pi Pico.

Example command: ./mk -b pico-serprog

Example command: ./mk -b stm32-vserprog

This also uses rom.sh as with the coreboot image build logic. It's all defined in that file, so read the main section pertaining to this file.

include/vendor.sh

Helper functions for downloading and injecting vendor files. How to use:

./mk inject ARGUMENTS
./mk -d coreboot TARGET

Refer elsewhere in the documentation for how to handle vendor files.

script/

script/trees

This is the other beating heart of Canoeboot. Used heavily by Canoeboot, this script is what handles defconfig files for SeaBIOS, U-Boot and coreboot; it used to be separate scripts, but the logic was unified under this single script.

It also handles simple git trees, where there is only one revision for the project, e.g. GRUB, and the command syntax is the same. Whether a project is multi-tree or single-tree is determined by the presence of the file config/PROJECT/build.list - if it exists, it's multi-tree, otherwise single-tree.

It also, in addition to downloading from git, can handle modification or updating of defconfig files. As already stated, and stated further: it is Canoeboot's other beating heart. Break this, and you break Canoeboot.

For multi-tree projects, it handles the following files (PROJECT can be coreboot, seabios or u-boot):

config/PROJECT/build.list (defines what files to copy, after building for the target)
config/PROJECT/*/target.cfg (cbmk build parameters, project project/target)
config/PROJECT/*/config/* (defconfig files)

For single-tree projects, these files are used:

config/git/ - files are concatenated and then scanned, to find project info.

NOTE: For multi-tree projects, config/git is still used, to download the upstream repository to src/PROJECT/PROJECT but with git revision being HEAD. In this way, you always have the latest code, but revisions defined in config/PROJECT/TARGET/target.cfg will define a tree, then config/PROJECT/TREE/target.cfg (which could be the same as TARGET, but this is not the preferred style in cbmk) will define a revision; then, the directory src/PROJECT/TREE will be created, reset to the specific revision - for multi-tree projects, all defined targets are scanned for their corresponding tree, and the trees are prepared as defined above.

Basic command: ./mk FLAG projectname

Special operation: for building coreboot utilities cbfstool and ifdtool to go under cbutils/, do this:

./mk -d coreboot TREENAME

Or define specific coreboot tree such as:

./mk -d coreboot default
./mk -d coreboot cros

FLAG values are (only one to be used at a time):

-b builds an image for the target, based on defconfig for multi-tree projects, or based only on a Makefile for single-tree projects; on some single-tree projects, this script also handles cmake.
-u runs make oldconfig on the target's corresponding source tree, using its defconfig (useful for automatically updating configs, when updating trees like when adding patches or switching git revisions)
-m runs make menuconfig on the target's corresponding source tree, using its defconfig (useful for modifying configs, e.g. changing CBFS size on a coreboot image)
-c tries make distclean, deferring to make clean under fault conditions and from that, non-zero exit under fault conditions. This is done on the target's corresponding source tree.
-x tries 'make crossgcc-clean`. This only works on coreboot trees, but no error status will be returned on exit if you try it on other project trees; no action will be performed.
-f downloads the Git repository for the given project, and resets to a revision as defined under config/git/, or (for multi-tree projects), the file config/PROJECT/TREE/target.cfg to create src/project/treename.

As for *projectname", this can either be coreboot, u-boot or seabios.

Example commands:

./mk -b coreboot
./mk -b coreboot x200_8mb
./mk -b coreboot x200_8mb x60
./mk -x coreboot default
./mk -u seabios
./mk -m u-boot gru_bob
./mk -f coreboot
./mk -d coreboot default
./mk -d coreboot

NOTE: the -x and -c options will cause an exit with zero status, when the target's corresponding source tree is unavailable; a non-zero status is only return under fault conditions when said source tree is available. ALL other flags will cause the very same source tree to be downloaded and prepared, if unavailable and that too will return with non-zero status under fault conditions.

NOTE: "target" can indeed be the tree name, under some circumstances. For example, ./mk -m seabios default

After projectname, a target can be specified, but if no target is specified, then all targets will be operated on. For example, ./mk -b coreboot will attempt to build all coreboot ROM images.

NOTE: the coreboot projectname here shall cause the ROM images to go under elf/ - this is the no-payload ROM images, which are later used separately by script/build/roms to provide full images, with payloads inserted. It is an intentional design choice of Canoeboot, to split it up this way and not use coreboot's own build system to handle payloads.

In cbmk, there are two types of git download: simple downloads where only a single revision would ever be used, or multi downloads where different revisions are used depending on target.

All such downloads are simple downloads, except for coreboot, U-Boot and SeaBIOS which are multi downloads. The other requirement is that defconfigs be used, though this could be worked around in the future if a multi setup is needed on a project that does not use defconfigs (this is not yet the case in cbmk).

All of this used to about 20 different scripts, all with much-duplicated logic. Now it is unified, efficiently, under a single script.

Remember: code equals bugs, so less code equals fewer bugs.

index.md 55 KB Cronologia Originale

Automated coreboot build system

Best practises for learning cbmk

Don't be deceived by simplicity

Environmental variables

XBMK_THREADS

XBMK_RELEASE

Projects/files downloaded/generated by cbmk

bin/

cache/

cache/app

cache/file/

cache/hash

cache/repo

elf/

release/

src/

src/coreboot/

src/flashprog/

src/grub/TREE

src/memtest86plus/

src/seabios/

src/u-boot/

src/pico-serprog

src/stm32-vserprog

tmp/

util/

util/dell-flash-unlock/

util/nvmutil/

util/spkmodem_recv/

config/

config/PROJECT*/nuke.list

config/coreboot

config/coreboot/BOARDNAME/

config/coreboot/BOARDNAME/patches/

config/coreboot/BOARDNAME/target.cfg

config/coreboot/BOARDNAME/config/

config/dependencies/

config/git/

config/data/PROJECT/mkhelper.cfg

GRUB config

config/data/grub/background

config/data/grub/background/background1024x768.png

config/data/grub/background/background1280x800.png

config/data/grub/background/COPYING

config/grub/TREE/config/

config/grub/config/AUTHORS

config/grub/config/COPYING

config/grub/TREE/config/payload

config/data/grub/memdisk.cfg

config/data/grub/keymap/

config/data/grub/keymap/*.gkb

config/data/grub/module/TREE

config/grub/TREE/patches/

config/ifd/*

config/seabios/

config/data/seabios/build.list

config/seabios/default/

config/seabios/default/config/

config/seabios/default/config/libgfxinit

config/seabios/default/config/normal

config/seabios/default/config/vgarom

config/seabios/default/target.cfg

config/u-boot/

config/data/u-boot/build.list

config/u-boot/TREENAME/

config/u-boot/TREENAME/patches/

config/u-boot/TREENAME/target.cfg

config/u-boot/TREENAME/config/

config/submodule

config/data/PROJECT

U-Boot build system

Config files in cbmk root directory

projectsite

projectname

version

versiondate

Scripts in root directory of cbmk

build

include/

index.md 55 KB

Cronologia Originale