The fwup utility is a configurable image-based firmware update utility for embedded Linux-based systems. It has two modes of operation. The first mode creates compressed archives containing root file system images, bootloaders, and other image material. These can be distributed via websites, email or update servers. The second mode applies the firmware images in a robust and repeatable way. The utility has the following features:

1. Uses standard ZIP archives to make debugging and transmission simple.

2. Simple, but flexible configuration language to enable firmware updates on various platforms and firmware update policies.

3. Streaming firmware update processing to simplify target storage requirements.

4. Multiple firmware update task options per archive so that one archive can upgrade varying target configurations

5. Basic disk partitioning and FAT filesystem manipulation

6. Human and machine readable progress.

7. Automatic detection of MMC and SDCards. This option queries the user before writing anything by default to avoid accidental overwrites.

8. Firmware archive digital signature creation and verification (BETA!!)

9. Permissive license (Apache 2.0 License - see end of doc)

This utility is based off of firmware update utilities I've written for various projects. It has already received a lot of use with the open source Nerves Project and other embedded projects. I tend to lock the version of the firmware update utility once embedded devices using it start leaving the lab so that I don't brick them with an upgrade. Once this project hits 1.0 I will avoid making backward incompatible changes. (I have actually only made a couple.) I do encourage you to use this utility, but please take care in upgrading fwup and test that your new fwup.conf files still work on devices with old versions of fwup. This seems like standard practice, but since bricking devices in the field is so painful, please take care.

See bbb-buildroot-fwup for firmware update examples for the BeagleBone Black and Raspberry Pi. The Nerves Project has some similar examples. The regression tests can also be helpful.

My real world use of fwup involves writing the new firmware to place on the Flash that's not in current use and then 'flipping' over to it at the very end. The examples tend to reflect that. fwup can also be used to overwrite the an installation in place assuming you're using an initramfs, but that doesn't give protection against someone pulling power at a bad time. Also, fwup's one pass over the archive feature means that firmware validation is mostly done on the fly, so you'll want to verify the archive first (see the -V option).

fwup requires libconfuse version 2.8. If you're running OSX, you're in luck. The latest version of libconfuse is in Homebrew:

brew install confuse

If you're on another platform, you almost certainly still have an older version (at least as of November 5th, 2015). Therefore, download and install libconfuse 2.8 or later. If you use an old version, the unit tests will fail due to environment variable substitution not working. (You'll understand if you skip this step.)

Install libarchive and libsodium. On Debian-based systems, run:

sudo apt-get install libarchive-dev libsodium-dev

On OSX, run:

brew install libarchive libsodium

Once that completes, clone or download the fwup source code and run the following:

./autogen.sh

# On Linux
./configure

# On OSX
CPPFLAGS="-I/usr/local/include -I/usr/local/opt/libarchive/include" LDFLAGS="-L/usr/local/lib -L/usr/local/opt/libarchive/lib" ./configure

make
sudo make install

The firmware update code is one of the parts of an embedded system where bugs can be frustratingly difficult or impossible to fix in the field. This project contains unit tests to reduce the risk. This doesn't remove all of the risk, and your project's fwup configuration file can certainly be buggy (e.g., bad flash offsets, etc.) so it is still important to test your firmware updates before deploying them.

To run the unit tests, you'll need the mtools package. This isn't used by fwup, but it's needed to verify that FAT file system operations work.

sudo apt-get install mtools

On OSX, you'll need mtools and a few other packages to run the unit tests:

brew install coreutils mtools gnu-sed

Then build the project as above and run:

make check

If the unit tests don't pass, please submit a bug report.

Usage: fwup [options]
  -a   Apply the firmware update
  -c   Create the firmware update
  -d <Device file for the memory card>
  -f <fwupdate.conf> Specify the firmware update configuration file
  -g Generate firmware signing keys (fwup-key.pub and fwup-key.priv)
  -i <input.fw> Specify the input firmware update file (Use - for stdin)
  -l   List the available tasks in a firmware update
  -m   Print metadata in the firmware update
  -n   Report numeric progress
  -o <output.fw> Specify the output file when creating an update (Use - for stdout)
  -q   Quiet
  -s <keyfile> A private key file for signing firmware updates
  -S Sign an existing firmware file (specify -i and -o)
  -t <task> Task to apply within the firmware update
  -v   Verbose
  -V Verify an existing firmware file (specify -i)
  -y   Accept automatically found memory card when applying a firmware update
  -z   Print the memory card that would be automatically detected and exit

Examples:

Create a firmware update archive:

  $ fwup -c -f fwupdate.conf -o myfirmware.fw

Apply the firmware update to /dev/sdc and specify the 'upgrade' task:

  $ fwup -a -d /dev/sdc -i myfirmware.fw -t upgrade

Generate a public/private key pair and sign a firmware archive:

  $ fwup -g
  (Store fwup-key.priv is a safe place. Store fwup-key.pub on the target)
  $ fwup -S -s fwup-key.priv -i myfirmware.fw -o signedfirmware.fw

fwup uses the Unix configuration library, libconfuse, so its configuration has some similarities to other programs. The configuration file is used to create firmware archives. During creation, fwup embeds a processed version of the configuration file into the archive that has been stripped of comments, has had all variables resolved, and has some additional useful metadata added. Configuration files are organized into scoped blocks and options are set using a key = value syntax.

For integration into build systems and other scripts, fwup performs environment variable substitution inside of the configuration files. Keep in mind that environment variables are resolved on the host during firmware update creation. Environment variables are referenced as follows:

key = ${ANY_ENVIRONMENT_VARIABLE}

It is possible to provide default values for environment variables using the :- symbol:

key = ${ANY_ENVIRONMENT_VARIABLE:-adefault}

Inside configuration files, it can be useful to define constants that are used throughout the file. All constants are stored as environment variables. By default, definitions do not overwrite environment variables with the same name:

define(MY_CONSTANT, 5)

To define a constant that does overrides the environment, use define!:

define!(MY_CONSTANT, "Can't override this")

At the global scope, the following options are available:

After setting the above options, it is necessary to create scopes for other options. The currently available scopes are:

A file-resource specifies a file on the host that should be included in the archive. Each file-resource should be given a unique name so that it can be referred to by other parts of the update configuration. The fwup will automatically record the length and BLAKE2b-256 hash of the file in the archive. These fields are used internally to compute progress and verify the contents of the archive. A typical file-resource section looks like this:

file-resource zImage {
        host-path = "output/images/zImage"
}

Resources are usually stored in the data directory of the firmware archive. This is transparent for most users. If you need to make the .fw file interoperate with other software, it is sometimes useful to embed a file into the archive at another location. This can be done by specifying an absolute path resource as follows:

file-resource "/my_custom_metadata" {
        host-path = "path/to/my_custom_metadata_file"
}

A mbr section specifies the contents of the Master Boot Record on the destination media. This section contains the partition table that's read by Linux and the bootloaders for finding the file systems that exist on the media. In comparison to a tool like fdisk, fwup only supports simplistic partition setup, but this is sufficient for many devices. Tools such as fdisk can be used to determine the block offsets and sizes of partitions for the configuration file. Offsets and sizes are given in 512 byte blocks. Here's a potential mbr definition:

mbr mbr-a {
        bootstrap-code-host-path = "path/to/bootstrap-data" # should be 440 bytes

        partition 0 {
                block-offset = ${BOOT_PART_OFFSET}
                block-count = ${BOOT_PART_COUNT}
                type = 0x1 # FAT12
                boot = true
        }
        partition 1 {
                block-offset = ${ROOTFS_A_PART_OFFSET}
                block-count = ${ROOTFS_A_PART_COUNT}
                type = 0x83 # Linux
        }
        partition 2 {
                block-offset = ${ROOTFS_B_PART_OFFSET}
                block-count = ${ROOTFS_B_PART_COUNT}
                type = 0x83 # Linux
        }
        partition 3 {
                block-offset = ${APP_PART_OFFSET}
                block-count = ${APP_PART_COUNT}
                type = 0x83 # Linux
        }
}

If you're using an Intel Edison or similar platform, fwup supports generation of the OSIP header in the MBR. This header provides information for where to load the bootloader (e.g.., U-Boot) in memory. The include-osip option controls whether the header is generated. The OSIP and image record (OSII) option names map directly to the header fields with the exception that length, checksum and image count fields are automatically calculated. The following is an example that shows all of the options:

mbr mbr-a {
    include-osip = true
    osip-major = 1
    osip-minor = 0
    osip-num-pointers = 1

    osii 0 {
        os-major = 0
        os-minor = 0
        start-block-offset = ${UBOOT_OFFSET}
        ddr-load-address = 0x01100000
        entry-point = 0x01101000
        image-size-blocks = 0x0000c000
        attribute = 0x0f
    }

    partition 0 {
        block-offset = ${ROOTFS_A_PART_OFFSET}
        block-count = ${ROOTFS_A_PART_COUNT}
        type = 0x83 # Linux
    }
}

The task section specifies a firmware update task. These sections are the main part of the firmware update archive since they describe the conditions upon which an update is applied and the steps to apply the update. Each task section must have a unique name, but when searching for a task, the firmware update tool only does a prefix match. This lets you define multiple tasks that can be evaluated based on conditions on the target hardware. The first matching task is the one that gets applied. This can be useful if the upgrade process is different based on the version of firmware currently on the target, the target architecture, etc. The following table lists the supported constraints:

Many more constraints to be added as needed

Each task can have options to change how it is applied:

The remainder of the task section is a list of event handlers. Event handlers are organized as scopes. An event handler matches during the application of a firmware update when an event occurs. Events include initialization, completion, errors, and files being decompressed from the archive. Since archives are processed in a streaming manner, the order of events is deterministice based on the order that files were added to the archive. If it is important that one event happen before another, make sure that file-resource sections are specified in the desired order. The following table lists supported events:

The event scopes contain a list of actions. Actions can format file systems, copy files to file systems or write to raw locations on the destination.

Firmware archives can be authenticated using a simple public/private key scheme. To get started, create a public/private key pair by invoking fwup -g. The algorithm used is Ed25519. This generates two file: fwup-key.pub and fwup-key.priv. It is critical to keep the signing key, fwup-key.priv secret.

To sign an archive, pass -s fwup-key.priv to fwup when creating the firmware. The other option is to sign the firmware archive after creation with --sign or -S.

To verify that an archive has been signed, pass -p fwup-key.pub on the command line to any of the commands that read the archive. E.g., -a, -l or -m.

It is important to understand how verification works so that the security of the archive isn't compromised. Firmware updates are apply in one pass to avoid needing a lot of memory or disk space. The consequence of this is that verification is done on the fly. The main metadata for the archive is always verified before any operations occur. Cryptographic hashs (using the BLAKE2b-256 algorithm) of each file contained in the archive is stored in the metadata. The hash for each file is computed on the fly, so a compromised file may not be detected until it has been written to Flash. Since this is obviously bad, the strategy for creating firmware updates is to write them to an unused location first and then switch over at the last possible minute. This is desirable to do anyway, since this strategy also provides some protection against the user disconnecting power midway through the firmware update.

The Nerves has examples for the Beaglebone Black, Raspberry Pi, and a couple x86 platforms. See the board subdirectory in the source tree.

The scenario is that you need to store some metadata or some other information that is useful to another program. For example, you'd like to include some documentation or other notes inside the firmware update archive that the destination device can pull out and present on a UI. To do this, just add file-resource blocks for each file. These blocks don't need to be referenced by an on-resource block.

If you are using git, you can invoke fwup as follows:

GITDESCRIBE=`git describe` fwup -c -f myupdate.conf -o out.fw

Then in myupdate.conf add the line:

meta-version = "${GITDESCRIBE}"

On the target device, you can retrieve the version by using -m. For example:

$ fwup -m -i out.fw
meta-product = "Awesome product"
meta-description = "A description"
meta-version = "v0.0.1"
meta-author = "Me"
meta-platform = "bbb"
meta-architecture = "arm"
meta-creation-date = "2014-09-07T19:50:57Z"

In general, fwup is written to write to Flash memory in large blocks so that the update can occur quickly. Obviously, reducing the amount that needs to get written always helps. Beyond that, most optimizations are platform dependent. Linux caches writes so aggressively that writes to Flash memory are nearly as fast as possible. OSX, on the other hand, does very little caching, so doing things like only working with one FAT filesystem at a time can help. In this case, fwup only caches writes to one FAT filesystem at a time, so mixing them will flush caches. OSX also is very slow to unmount disks, so keep in mind that performance can only be so fast on some systems.

This utility contains source code with various licenses. The bulk of the code is licensed with the Apache 2.0 license which can be found in the LICENSE file.

The FAT filesystem code (FatFs) comes from http://elm-chan.org/fsw/ff/00index_e.html and has the following license:

FatFs module is a generic FAT file system module for small embedded systems.
This is a free software that opened for education, research and commercial
developments under license policy of following terms.

 Copyright (C) 2014, ChaN, all right reserved.

* The FatFs module is a free software and there is NO WARRANTY.
* No restriction on use. You can use, modify and redistribute it for
  personal, non-profit or commercial products UNDER YOUR RESPONSIBILITY.
* Redistributions of source code must retain the above copyright notice.

On systems without the function open_memstream(), code from http://piumarta.com/software/memstream/ is included. It is distributed under the MIT license