Code Monkey home page Code Monkey logo

ubuntu-remaster-bbb's Introduction

About

This script can do multiple things to ubuntu iso images:

  • make casper accept extra filesystems to loopback-boot from, e.g. f2fs or exfat
  • add kernel modules from the squashed filesystem to the initrd, for example, to support the above mentioned filesystems
  • remove nvidia drivers
  • remove tpm stuff (the enhanced-secureboot layers introduced in 23.10)

When processing an image, it does not mount anything, not even with FUSE or udisks. It is so by design to work under unprivileged docker.

It works with ubuntu flavours as well. As for other ubuntu-based images, it should generally work, but you might need to give it a hint on the release version with the --distro option. The version is guessed from /.disk/info file in the image. For example, 22.04-based KDE neon does not mention the numeric version there. In the case, you could fix it with --distro 22.04.

Dependencies

The script checks if required binaries are available on the fly. The full set of packages to have pre-installed is as such (you likely have some already): xorriso, xxd, file, cpio, squashfs-tools, kmod, lz4, xz-utils, zstd, pv. Also, unmkinitramfs-turbo should be installed into PATH.

Usage

ubuntu-remaster-bbb [options] input output

Supported iso versions: 14.04 to 24.04 for desktop, 18.04 to 24.04 for live-server.

Below list stands for a comma separated list of values.

Options:

-h, --help
Show usage
--fs list
Make casper accept the filesystems from the list to loopback-boot from and preload corresponding kernel modules. You can use it multiple times
--module list
Add modules from the list to the initrd. Prepend optional modules with tilde. You can use it multiple times
--no-nvidia
Remove nvidia drivers
--no-tpm
Remove tpm stuff (the enhanced-secureboot layers introduced in 23.10)
--distro xx.yy
Give a hint on ubuntu release number for the iso image. Usable for ubuntu-based images which dont set the version number in /.disk/info, like KDE neon
--tmp-dir path
Set a custom dir for temp files. By default, $TMPDIR value is used, if set and not empty, or /tmp otherwise
--dry-run
Stop on the final step (reassembling the iso) and print xorriso options as a bash array. This can be used along with --no-cleanup to mix in custom modifications
--no-cleanup
Do not remove temp files on exit
--hwe-only
Only modify the hwe initrd. By default, both initrd are processed if there are two. In some cases it is not possible, for example, when making 18.04 live-server iso bootable from exfat, since only the hwe modules squash contains the exfat module

Along with the script, there are two symlinks pointing to it: ubuntu-remaster-f2fs and ubuntu-remaster-exfat. When called by the symlinks, such options are implied by the script:

ubuntu-remaster-f2fs
--fs f2fs --module f2fs,~crc32_generic,~crc32-pclmul
ubuntu-remaster-exfat
--fs exfat --module exfat

Sample run

ubuntu-23.10.1-desktop-amd64.iso: single initrd; only patch scripts

> ubuntu-remaster-f2fs --tmp-dir /run/shm ubuntu-23.10.1-desktop-amd64.iso /run/shm/mantic.iso
patch casper/initrd
| main archive offset=85893120, compression=zstd
| extract the main archive into /run/shm/remaster.223586.initrd_main
| patch casper scripts for fs support
| | f2fs
| | | casper-helpers:is_supported_fs()
| | | casper-helpers:wait_for_devs()
| kernel 6.5.0-9-generic
| modules archive offset=7278080, size=78615040
| extract the modules archive into /run/shm/remaster.223586.initrd_early3
| checkout kernel modules
| | skip f2fs (included)
| | skip crc32_generic (included)
| | skip crc32-pclmul (included)
| compress new initrd with zstd
[==============================================================================>] 100%

xorriso 1.5.4 : RockRidge filesystem manipulator, libburnia project.

[long xorriso log]

Writing to '/run/shm/mantic.iso' completed successfully.

ubuntu-18.04.6-live-server-amd64.iso: two initrd; patch scripts, add missing kernel modules

> ubuntu-remaster-f2fs --tmp-dir /run/shm ubuntu-18.04.6-live-server-amd64.iso /run/shm/bionic.iso
patch casper/hwe-initrd
| main archive offset=4641792, compression=gzip
| extract the main archive into /run/shm/remaster.224008.hwe-initrd_main
| patch casper scripts for fs support
| | f2fs
| | | lupin-helpers:is_supported_fs()
| | | lupin-helpers:wait_for_devs()
| kernel 5.4.0-84-generic
| checkout kernel modules
| | add f2fs
| | add crc32_generic
| | add crc32-pclmul
| compress new initrd with gzip
[==============================================================================>] 100%
patch casper/initrd
| main archive offset=4641792, compression=gzip
| extract the main archive into /run/shm/remaster.224008.initrd_main
| patch casper scripts for fs support
| | f2fs
| | | lupin-helpers:is_supported_fs()
| | | lupin-helpers:wait_for_devs()
| kernel 4.15.0-156-generic
| checkout kernel modules
| | add f2fs
| | add crc32_generic
| | add crc32-pclmul
| compress new initrd with gzip
[==============================================================================>] 100%

xorriso 1.5.4 : RockRidge filesystem manipulator, libburnia project.

[long xorriso log]

Writing to '/run/shm/bionic.iso' completed successfully.

Sample grub setup for f2fs/exfat

Here we assume /dev/sdX is some flash drive with such GPT partitions (type and desc in gdisk terms):

sizetypedescfsmount point
1MEF02BIOS boot partition
40MEF00EFI system partitionvfat/mnt/sdX2
rest8300Linux filesystemf2fs OR exfat/mnt/sdX3

Notice: the f2fs partition should be formatted with default settings. The recommended way is to enable the checksums support, but grub’s f2fs driver does not cope with such features.

i386-pc target

sudo grub-install --target i386-pc --boot-directory /mnt/sdX3/boot /dev/sdX

x86_64-efi target

Signed prebuilt grub images from grub-efi-amd64-signed package do not bundle f2fs and exfat modules (as of Q1 2024), so we have to make grub NOT use the images for sure. This way it would not work out-of-the-box when secure boot is enabled, but that is out of scope for this doc.

sudo grub-install --target x86_64-efi --boot-directory /mnt/sdX3/boot \
     --efi-directory /mnt/sdX2 --removable --no-uefi-secure-boot /dev/sdX

Next, put grub.cfg from this repo into /mnt/sdX3/boot/grub/. The config assumes the iso images are under /mnt/sdX3/boot/iso/ AND contain ”buntu” in their names.

You could get more elaborate grub configs from such projects as GLIM (ubuntu support there is not perfect though).

Docker image

Notice: the Dockerfile uses anvanced syntax. You may need to install docker-buildx package.

You can build the image like this (upon entering the repo dir):

docker build -t ubuntu-remaster-bbb .

The entry point is set to the script.

Docker wrapper

ubuntu-remaster-bbb.docker makes it easy to run the docker image. It mounts the input file (read only) and output dir into the container. ubuntu-remaster-bbb is the assumed name for the docker image.

If you start it with sudo, the script in the container runs under your pre-sudo uid:gid (as per $SUDO_UID and $SUDO_GID env vars). Otherwise, it runs under your effective uid:gid.

Without any args (or with --help option), the wrapper prints some usage text. --script-help option can be used to pass --help to the script. Otherwise, such form is expected (notice the options must be delimited with -- from the rest):

ubuntu-remaster-bbb.docker [options] -- input output

Tech details

Ubuntu casper-based iso images (at least since 10.04 for desktop and since 18.04 for live server) can boot from the iso file as-is, provided its path with iso-scan/filename kernel arg. Minimal sample grub config assuming the iso is stored in /boot/iso on the same filesystem where /boot/grub is located:

menuentry "ubuntu-22.04.4 desktop" /boot/iso/ubuntu-22.04.4-desktop-amd64.iso {
    loopback loop "$2"
    linux (loop)/casper/vmlinuz boot=casper iso-scan/filename="$2"
    initrd (loop)/casper/initrd
}

It just works as long as the filesystem is recognized by casper. Speaking about f2fs, initrd in ubuntu iso images bundles the driver since 20.04, but casper itself does not include f2fs into its list of supported filesystems (as of Q1 2024):

is_supported_fs(){
    [ -z "${1}" ] && return 1
    case ${1} in
        ext2|ext3|ext4|xfs|jfs|reiserfs|vfat|ntfs|iso9660|btrfs|udf)
            return 0
            ;;
    esac
    return 1
}

and does not preload the f2fs kernel module. The exfat module is not even in initrd.

ubuntu-remaster-bbb's People

Contributors

slowpeek avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar

Forkers

soleagle

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.