robertdigital / skiffos Goto Github PK

SkiffOS is a lightweight operating system for any Linux-compatible computer, ranging from RPi, Odroid, NVIDIA Jetson, to Desktop PCs, Laptops (i.e. Apple MacBook), Phones, Cloud VMs, and more. It is:

• Familiar: uses simple Makefile and KConfig language for configuration.
• Flexible: supports any OS distribution inside containers w/ ssh drop-in.
• Portable: replicate the exact same system across any hardware or arch.
• Reliable: read-only minimal in-RAM host system boots reliably every time.
• Reproducible: offline and deterministic builds for reproducible behavior.

SkiffOS adds a configuration layering system to the Buildroot cross-compiler, which makes it easy to re-target applications to new hardware. Layers are merged together as specified in the SKIFF_CONFIG comma-separated environment variable. As a basic example: SKIFF_CONFIG=pi/4,core/gentoo starts Gentoo on a Raspberry Pi 4 in a Docker container.

The default configuration produces a minimal (~100Mb) in-RAM host OS with SSH and network connectivity, and includes a comprehensive set of debug tools. The host OS can be easily remotely updated with the push_image script, using rsync.

The "skiff/core" layer enables Docker ("apps/docker") and a default environment based on Ubuntu with a full graphical desktop environment. Others including "core/gentoo" and "core/dietpi" are available.

Most Linux devices have a unique set of requirements for kernel, firmware, and hardware support packages. The SkiffOS host OS separates hardware-specific support from the containerized user environments, simplifying management of updates across multiple hardware combinations.

The SKIFF_CONFIG comma-separated environment variable selects which configuration layers should be merged together to configure the build.

This example uses pi/4, which can be replaced with any of the hardware support packages listed in the Supported Systems table.

$ make                             # lists all available layers
$ export SKIFF_WORKSPACE=default   # optional: supports multiple SKIFF_CONFIG at once
$ export SKIFF_CONFIG=pi/4,skiff/core
$ make configure                   # configure the system
$ make compile                     # build the system

After you run make configure your SKIFF_CONFIG selection will be saved and automatically restored in future sessions. The compile command builds the OS.

The optional SKIFF_WORKSPACE variable defaults to default, but is useful for compiling multiple SKIFF_CONFIG simultaneously. Each workspace is isolated from the others and can have a completely different configuration. The build can be interrupted and resumed with make compile as needed.

You will need a SSH public key to access the system. If you don't have one, create a SSH key on your development machine. Add the public key (usually located at ~/.ssh/id_rsa.pub) to your build by copying it to overrides/root_overlay/etc/skiff/authorized_keys/my-key.pub. The keys can also be added to a configuration layer for future use.

There are many other utility commands made available by Buildroot, which can be listed using make br/help, some examples:

$ make br/menuconfig # optionally explore Buildroot config
$ make br/sdk        # build relocatable SDK for target
$ make br/graph-size # graph the target packages sizes

You can add apps/portainer to SKIFF_CONFIG to enable the Portainer UI.

Once the build is complete, it's time to flash the system to a SD card. You will need to switch to sudo bash for this on most systems.

$ sudo bash             # switch to root
$ blkid                 # look for your SD card's device file
$ export PI_SD=/dev/sdz # make sure this is right!
$ make cmd/pi/common/format  # tell skiff to format the device
$ make cmd/pi/common/install # tell skiff to install the os

The device needs to be formatted only one time, after which, the install command can be used to update the SkiffOS images without clearing the persistent data. The persist partition is not touched in this step, so anything you save there, including all Docker containers and system configuration, will not be modified.

Connect using SSH to root@my-ip-address to access the SkiffOS system, and connect to core@my-ip-address to access the "Core" system container. See the section above about SSH public keys if you get a password prompt.

The mapping between users and containers can be edited in the /mnt/persist/skiff/core/config.yaml file.

The system can then be upgraded over-the-air (OTA) using the rsync script:

$ ./scripts/push_image.bash root@my-ip-address

The SkiffOS upgrade (or downgrade) will take effect on next reboot.

SkiffOS is based on Buildroot, which can compile operating systems for any Linux-compatible machine.

Here are the boards/systems currently supported:

All targets marked "tested" use automated end-to-end testing on real hardware. Targets marked "Obsolete" are discontinued by their manufacturer but still have a corresponding SkiffOS configuration and should still work.

Adding support for a board involves creating a Skiff configuration package for the board, as described above. If you have a device that is not yet supported by SkiffOS, please open an issue.

The Skiff Core subsystem, enabled with the skiff/core layer or by selecting any of the core environment packages, automatically configures mappings between users and containerized environments. It maps incoming SSH sessions accordingly:

• Configured using a YAML configuration file skiff-core.yaml.
• The container image is either pulled or built from scratch.
• systemd and/or other init systems operate as PID 1 inside the container.

This allows virtually any workflow to be migrated to Skiff. The config file structure is flexible, and allows for any number of containers, users, and images to be defined and built.

Any existing GNU/Linux system with compatibility with the running kernel version can be loaded as a Docker image with the docker import command.

All core configurations work with all target platforms.

The primary distributions and images supported are:

Other less frequently updated images:

The default configuration creates a user named "core" mapped into a container, but this can be adjusted with the skiff-core.yaml configuration file:

containers:
  core:
    image: skiffos/skiff-core-gentoo:latest
    [...]
users:
  core:
    container: core
    containerUser: core
    [...]

The full example config is in the skiff/core package.

To customize a running system, edit /mnt/persist/skiff/core/config.yaml and run systemctl restart skiff-core to apply. You may need to delete existing containers and restart skiff-core to re-create them after changing their config.

The config format is defined in the skiff-core repo.

There are three release channels: next, master, and stable.

Skiff can be upgraded or downgraded (rolled back) independently from the persistent storage partition. This allows for easy OTA, and significant improvements in confidence when upgrading system components.

Skiff supports modular configuration layers. A configuration directory contains kernel configs, buildroot configs, system overlays, and misc. files.

Layers are named as namespace/name. For example, a Raspberry Pi 4 configuration would be pi/4 and Docker is apps/docker.

├── cflags:         compiler flags in files
├── buildroot:      buildroot configuration fragments
├── buildroot_ext:  buildroot extensions (extra packages)
├── buildroot_patches: extra Buildroot global patches
│   ├── <packagename>: patch files for Buildroot <packagename>
│   └── <packagename>/<version>: patches for package version
├── extensions:     extra commands to add to the build system
│   └── Makefile
├── hooks:          scripts hooking pre/post build steps
│   ├── post.sh
│   └── pre.sh
├── kernel:         kernel configuration fragments
├── kernel_patches: kernel .patch files
├── root_overlay:   root overlay files
├── metadata:       metadata files
│   ├── commands
│   ├── dependencies
│   ├── description
│   └── unlisted
├── resources:     files used by the configuration package
├── scripts:       any scripts used by the extensions
├── uboot:         u-boot configuration fragments
└── uboot_patches: u-boot .patch files

All files are optional.

You can set the following env variables to control this process:

• SKIFF_CONFIG_PATH_ODROID_XU: Set the path for the ODROID_XU config package. You can set this to add new packages or override old ones.
• SKIFF_EXTRA_CONFIGS_PATH: Colon : separated list of paths to look for config packages.
• SKIFF_CONFIG: Name of skiff config to use, or comma separated list to overlay, with the later options taking precedence

These packages will be available in the Skiff system.

It's often useful to be able to adjust the configs locally during development without actually creating a new configuration layer. This can be easily done with the overrides layer.

The overrides directory is treated as an additional configuration layer. The layout of the configuration layers is described above. Overrides is ignored by Git, and serves as a quick and easy way to modify the configuration.

To apply the changes & re-pack the build, run "make configure compile" again.

Workspaces allow you to configure and compile multiple systems at a time.

Set SKIFF_WORKSPACE to the name of the workspace you want to use. The Buildroot setup will be constructed in workspaces/$SKIFF_WORKSPACE. You can also place configuration files in overrides/workspaces/$SKIFF_WORKSPACE/ to override settings for that particular workspace locally.

The virt/ packages are designed for running Skiff in various virtualized environments.

Here is a minimal working example of running Skiff in Qemu:

$ SKIFF_CONFIG=virt/qemu make configure compile
$ make cmd/virt/qemu/run

Here is a minimal working example of running Skiff in Docker:

$ SKIFF_CONFIG=virt/docker,skiff/core make configure compile
$ make cmd/virt/docker/buildimage
$ make cmd/virt/docker/run

# inside container
$ su - core

The build command compiles the image, and run executes it.

You can execute a shell inside the container with:

$ make cmd/virt/docker/exec
# alternatively
$ docker exec -it skiff sh

Alternatively, run the latest demo release on Docker Hub:

docker run -t -d --name=skiff \
  --privileged \
  --cap-add=NET_ADMIN \
  --security-opt seccomp=unconfined \
  --stop-signal=SIGRTMIN+3 \
  -v /sys/fs/cgroup:/sys/fs/cgroup:ro \
  -v $(pwd)/skiff-persist:/mnt/persist \
  skiffos/skiffos:latest

SkiffOS can be configured dynamically with files in the "persist" partition.

Set the hostname by placing the desired hostname in the skiff/hostname file on the persist partition. You could also set this in one of your config packages by writing the desired hostname to /etc/hostname.

Network configurations are loaded from /etc/NetworkManager/system-connections or from the persist partition at skiff/connections.

You can use nmcli on the device to manage NetworkManager, and any connection definitions written by nmcli device wifi connect or similar will automatically be written to the persist partition and persisted to future boots.

To connect to WiFi: nmcli device wifi connect myssid password mypassword.

The configuration file format for these connections is documented here with examples.

The system will generate the authorized_keys file for the users on startup.

It takes SSH public key files (*.pub) from these locations:

• /etc/skiff/authorized_keys from inside the image
• skiff/keys from inside the persist partition

Your SSH public key will usually be located at ~/.ssh/id_rsa.pub.

To mount a Linux disk, for example an ext4 partition, to a path inside a Docker container, you can use the Docker Volumes feature:

# create a volume for the storage drive
docker volume create --driver=local --opt device=/dev/disk/by-label/storage storage

# run a temporary container to view the contents
docker run --rm -it -v storage:/storage --workdir /storage alpine:edge sh

The volume can be mounted into a Skiff Core container by adding to the mounts list in /mnt/persist/skiff/core/config.yaml:

containers:
  core:
    image: skiffos/skiff-core-gentoo:latest
    mounts:
      - storage:/mnt/storage

After adding the mount, delete and re-create the container:

docker rm -f core
systemctl restart skiff-core

SkiffOS is built & supported by Aperture Robotics, LLC.

Community contributions and discussion are welcomed!

Please file a GitHub issue and/or Join Discord with any questions.

... or feel free to reach out on Matrix Chat!