resflash

resflash

Resilient images for flashing operating systems

Resflash is a tool for building scriptable OpenBSD images for embedded and cloud platforms with easy, single-file upgrades. Resflash uses read-only and memory-backed filesystems, and because the filesystems are written to only during system upgrades, they are not subject to corruption or fsck due to power loss - and even cheap USB flash drives can last virtually forever. Resflash images can be written to any bootable media (flash, SSD, or spinning hard drive) and make great boot drives for firewalls, NAS, or VM servers. Resflash was written from scratch, with inspiration drawn from NanoBSD and flashrd.

Features

• Read-only filesystems on all disk-backed partitions. Power can be safely lost at any time.
• An easy, single-file upgrade process.
• Persistent configuration changes are supported by a /cfg partition, stored either manually or on shutdown, and re-populated on boot.
• Full local and remote package support, at build time or using the standard package tools.
• Easy failover to the previous working image in the event of a boot failure (console access required). Think of it as an /altroot on steroids.
• Supports amd64 and i386.
• System requirements comparable to that of OpenBSD (2+ GB drive recommended).
• No additional packages required for build.

Features exclusive to resflash

• A one-command build process, with no configuration file or sector calculations required.
• An unmodified OpenBSD operating system - no custom kernels, no ramdisks.
• No proprietary file formats. Disk images and filesystems are mountable partitions and filesystems, respectively.
• Images can be built using only the OpenBSD distribution sets, no compiler or OpenBSD source is required.
• Supports UEFI and BIOS/CSM booting with the same image.
• Branch-agnostic: Build -current images on -release and vice versa.
• Arch-agnostic: Build i386 images on amd64 and vice versa (same-endian). Note: Cross-arch builds do not support package installation at build time.
• Files in /etc or /var can be configured for automatic saving - en masse, by directory, or by file (i.e. ., /etc/ssh, /var/db/dhc*).
• Supports fw_update and syspatch at build time.
• System upgrades update the MBR, biosboot(8), boot(8), and UEFI bootloader (where applicable).
• Supports xxHash checksums for both builds and upgrades, with graceful fallback to standard checksums.
• Supports building with ksh or Bash (but why?).

How does it work?

Resflash images contain two main data partitions, one active and one inactive. During the upgrade process, the inactive partition is updated, tested, and set active for the next boot. A /cfg partition can be used to store modifications from the mfs filesystems (/etc and /var) and are overlaid at boot time. Small /mbr and /efi partitions are used to maintain the BIOS and UEFI boot code, respectively.

Coverage

• BSD Now - Episode 099: BSD Gnow
• Hacker News - 20150813
• Firmware Security - 20150922
• lippmann wiki - 20210410

Downloads

• The latest release source can be found in the resflash master branch on GitLab. Resflash always supports the two currently-supported releases of OpenBSD, starting with 5.6. A .tar.gz download is also available.
• Premade -stable images are available at https://stable.rcesoftware.com/pub/resflash. Versions are available for the latest OpenBSD release on at least amd64 and i386, each with VGA or com0 console. Both .img and .fs files, for installs and upgrades, are available.
• Sets from the -stable branch are available at https://stable.rcesoftware.com/pub/OpenBSD for at least amd64 and i386. These are not built in an automated fashion, but should be updated shortly after an errata notice. Images can also be built using -release sets and the --syspatch flag.
• Sample tools for building and configuring resflash images can be found in the resflash-tools repository on GitLab.

Usage

1. Create an OpenBSD base directory with a minimum of the following:

   • bsd (sp or mp supported)
   • baseXY.tgz
   • (base dir)/var/sysmerge/etc.tgz

Sets must be unpacked as root using tar zxfph set.tgz.

1. Run build_resflash.sh:

   usage: build_resflash.sh [-binhV] [-p pkg_dir] [--pkg_list pkg1,pkg2]
                            [--pkg_path path1:path2] [-s com0_speed]
                            [--swap reserve_swap_in_mb] [--syspatch]
                            img_size_in_mb openbsd_base_dir
   

2. Write the .img file (not the .fs file) to the drive: dd if=resflash-amd64-1906MB-com0_115200-20230126_2239.img of=/dev/rsd3c bs=1m

Sample usage

Sample usage with --pkg_list pkg1,pkg2:

# ./resflash/build_resflash.sh --pkg_list dhcpcd-- -s 115200 1906 /usr/rdest
resflash 7.2.2

Validating OpenBSD base dir: /usr/rdest
Creating disk image: resflash-amd64-1906MB-com0_115200-20230126_2239.img
Creating filesystem: resflash-amd64-1906MB-com0_115200-20230126_2239.fs
Populating filesystem and configuring fstab
  Running fw_update (14s)
  Installing packages: https://cdn.openbsd.org/%m/ (3s)
Populating filesystem complete (28s)
Writing filesystem to disk image (13s)
Calculating filesystem checksum (6s)
Calculating disk image checksum (13s)
Build complete! (63s)

File sizes:
884M    resflash-amd64-1906MB-com0_115200-20230126_2239.fs
1.9G    resflash-amd64-1906MB-com0_115200-20230126_2239.img
Disk usage:
474M    resflash-amd64-1906MB-com0_115200-20230126_2239.fs
886M    resflash-amd64-1906MB-com0_115200-20230126_2239.img

Sample usage with --syspatch:

# ./resflash/build_resflash.sh -i -s 115200 --syspatch 2860 /usr/rdest
resflash 7.2.2

Validating OpenBSD base dir: /usr/rdest
Creating disk image: resflash-amd64-2860MB-com0_115200-20230126_2239.img
Creating filesystem: resflash-amd64-2860MB-com0_115200-20230126_2239.fs
Populating filesystem and configuring fstab
  Running fw_update: intel (4s)
  Relinking kernel relink and running syspatch (26s)
Populating filesystem complete (47s)
Writing filesystem to disk image (24s)
Calculating filesystem checksum (15s)
Calculating disk image checksum (24s)
Build complete! (114s)

File sizes:
1.3G    resflash-amd64-2860MB-com0_115200-20230126_2239.fs
2.8G    resflash-amd64-2860MB-com0_115200-20230126_2239.img
Disk usage:
1.2G    resflash-amd64-2860MB-com0_115200-20230126_2239.fs
1.3G    resflash-amd64-2860MB-com0_115200-20230126_2239.img

Upgrades

Unlike the initial installation, upgrades use .fs filesystem files. Upgrades take place by piping the .fs file through the /resflash/upgrade.sh script. This can be accomplished in many ways:

• The more secure, doas-enabled ssh way:
  1. On the build system, connect to the system to be upgraded: ssh -C root@10.0.x.y 'doas /resflash/upgrade.sh' < resflash-amd64-1906MB-com0_115200-20230126_2239.fs
  2. Review the output, confirm the filesystem checksum, and reboot.
• The less secure, trusted LAN-only way:
  1. On the system to be upgraded, run as root: nc -lv 1234|gzip -d|/resflash/upgrade.sh
  2. On the build system, run: gzip -1c resflash-amd64-1906MB-com0_115200-20230126_2239.fs|nc -Nv 10.0.x.y 1234
  3. Review the output, confirm the filesystem checksum, and reboot.

Sample output:

Writing filesystem to inactive partition
a31a54719bc64d484e0352a21918dffb66d51a510a1a00325b3ae7e6835455bd
Writing filesystem complete (44s)
Checking filesystem
/dev/sd0e (a3f53fb2e8e87988.e): 10727 files, 314270 used, 122017 free (825 frags,
15149 blocks, 0.2% fragmentation)
Overlaying data from /cfg/upgrade_overlay to filesystem
Updating fstab
Updating MBR
Updating biosboot(8) and boot(8)
Updating BOOTX64.EFI BOOTIA32.EFI bootloader(s)
Everything looks good, setting the new partition active
Upgrade complete! (48s)

Other build tools

• mount_resflash.sh - Mount all the partitions of a resflash .img or .fs file. This is useful for scripting configuration after a build.
• umount_resflash.sh - Unmount a mounted resflash .img or .fs file.

Host tools

• /etc/resflash.conf - Optional configuration file for automating backup of files in /etc or /var on shutdown. Consult the file for available options.
• /resflash/save_ssh_ike_soii_keys.sh - Save SSH, IKE, and SOII keys to /cfg.
• /resflash/set_root_pass.sh - Update root password and save necessary password db files to /cfg.
• /resflash/BUILD - Documents the version, build command, and build date used to create the image. Useful for keeping filesystem sizes consistent.

Problems?

Resflash is not a supported OpenBSD configuration. Please do not email bugs@ or misc@ asking for help. If you have a question or a bug to report, please post to our mailing list, submit an issue on GitLab, or email me directly.

Support OpenBSD

This project would not be possible without the work of the fine folks at OpenBSD. Please support them with a donation (1, 2) or purchase.

FAQ

What is the root password for the premade images?

Root passwords are generated dynamically at build time in the form 'resflashYYYYMMDD', where YYYYMMDD corresponds to the date value in the image filename. After logging in, run /resflash/set_root_pass.sh to set and persist a new root password to /cfg.

What is the difference between the .img and .fs files?

The .img files are disk images, including MBR partition tables, that are used for initial installation to a destination drive. The .fs files are filesystems that are used for in-place upgrades by /resflash/upgrade.sh.

How do I use the /cfg partition?

The /cfg partition is usually unmounted and stores modifications to the /etc and /var mfs filesystems. Files are saved either manually or on shutdown according to /etc/resflash.conf. To manually save a file, mount /cfg and then copy any file you want re-populated to /cfg/etc or /cfg/var, retaining the directory structure (i.e. /cfg/etc/hostname.em0 or /cfg/etc/ssh/sshd_config). Unmount /cfg when finished. You can also run /resflash/resflash.save manually to save configured files at any time.

Additionally, any directory hierarchy under /cfg/upgrade_overlay which will be copied over to the new root filesystem upon upgrade. This allows lightweight customization of the root filesystem (i.e. /root/.ssh/authorized_keys).

What about LBA and CHS?

Resflash requires an LBA-aware BIOS. CHS numbers have been bogus for 25 years, and I don't have the hardware for - or much interest in - supporting them. Make sure to set your Alix board to LBA mode.

Does the build_resflash.sh image size matter if I'm only building .fs files for upgrades?

Yes! Filesystem sizes are calculated from image size, so you will want to keep your image size consistent over the life of an image (see /resflash/BUILD if you forget). You can scale image size down without issue, but if you attempt to use a filesystem from a larger image for an upgrade, the filesystem will exceed the available space of the inactive partition, and the upgrade will fail.

Help! I ran the upgrade and now it won't boot. How do I fail over to the previous version?

At the OpenBSD boot prompt, enter set device hd0d and press enter, assuming that the 'e' partition is your upgraded partition that is failing to boot. If 'd' is failing, set it to hd0e. Before doing any diagnosis on your failed upgrade, you will want to mount /mbr and edit /mbr/etc/boot.conf to point to the working boot device.

Should I customize the .img file prior to writing to disk, or via /cfg?

There is no wrong answer here. If you're scripting your builds, it probably makes sense to use the (u)mount_resflash.sh tools to make all your changes to the .img or .fs directly, and then use /cfg exclusively for runtime files (i.e. /var/db/host.random). If you're using resflash for a single system, it's perfectly reasonable to save things like myname or hostname.em0 in /cfg/etc. If you need to modify files outside of /etc or /var, such as /root, that is best done via the .img or .fs file.