clearlinux / clr-boot-manager Goto Github PK
View Code? Open in Web Editor NEWKernel & Boot Loader Management
Home Page: https://clearlinux.org/
Kernel & Boot Loader Management
Home Page: https://clearlinux.org/
Main ticket items:
version
string for systemd-bootmachine-id
string for systemd-bootmachine-id
into target filenames on all systemsFor natively booted systems, utilise the machine-id
to ensure unique paths. This will ensure that CBM managed OSs can dual-boot with themselves (Dev/Production split)
For image mode builds, continue to use the current paths. This ensures that the first time CBM is run natively, it will perform a migration to the now-available machine-id
filenames.
To delete every kernel that not the one being used (or not the default of a given type).
First lines of the Dockerfile:
# Build container based on Clearlinux
FROM clearlinux:base
RUN swupd bundle-add os-clr-on-clr python3-basic
RUN swupd update
<snip>
When
$ sudo docker build -t clearlinux -f Dockerfile.clearlinux .
Sending build context to Docker daemon 177.7 kB
Step 1/8 : FROM clearlinux:base
---> cfe5f13b90b0
Step 2/8 : RUN swupd bundle-add os-clr-on-clr python3-basic
---> Using cache
---> 7bd1d6391ac4
Step 3/8 : RUN swupd update
---> Running in d271ffc27912
swupd-client software update 3.15.0
Copyright (C) 2012-2018 Intel Corporation
Update started.
Attempting to download version string to memory
Preparing to update from 21150 to 21520
Downloading packs...
.
Extracting go-basic pack for version 21420
Extracting koji pack for version 21500
Extracting kvm-host pack for version 21500
Extracting libX11client pack for version 21500
.
Extracting openssh-server pack for version 21330
.
Extracting package-utils pack for version 21320
Extracting perl-basic pack for version 21450
Extracting perl-basic-dev pack for version 21510
Extracting python2-basic pack for version 21510
Extracting python3-basic pack for version 21360
Extracting storage-utils pack for version 21360
.
Extracting web-server-basic pack for version 21450
Statistics for going from version 21150 to version 21520:
changed bundles : 30
new bundles : 0
deleted bundles : 0
changed files : 3410
new files : 5329
deleted files : 1658
Starting download of remaining update content. This may take a while...
File /usr/bin/FileCheck was not in a pack
........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Finishing download of update content...
Staging file content
File staging failed: /usr/include/clang/CrossTU
Calling post-update helper scripts.
none
[ERROR] cbm (../src/lib/files.c:L165): Invalid block device: /
[ERROR] cbm (../src/lib/system_stub.c:L31): Invalid block device: 0:75
Out of memory[ERROR] cbm (../src/bootman/update.c:L250): No kernels discovered in ///usr/lib/kernel, bailing
Update took 279.9 seconds
4981 files were not in a pack
The command '/bin/sh -c swupd update' returned a non-zero code: 18
I am a Solus user which uses clr-boot-manager for boot management. The user is expected to select the linux kernel to boot up during start up by spamming the space key (on UEFI hardware).
I have a laptop with UEFI setup. However, I cannot select the boot kernels by the above approach. My problem is that, the laptop becomes unresponsive once I hit the space key during startup and enter the page for selection of kernels. I can see which kernels are available but I cannot select any of them. Even worse, the laptop would freeze there with no response to any keystroke. It boots fine if I let it boot regularly to the log in page.
Because of this issue, I cannot switch the kernels on this laptop. I am proposing to add a command line option to clr-boot-manager so that I can choose what kernel to use in the next bootup. If it works fine, then I still can use clr-boot-manager report-booted
to keep this option. Otherwise the option would not be saved and in future startups the machine will use the original kernel it used. In addition, there can an option to list all available kernels.
One example of such a feature could be:
clr-boot-manager list-kernels -- list all kernels installed on the machine
clr-boot-manager select-kernel XXX -- select a kernel for next boot
After swupd deleted my current kernel's files from /usr/lib/kernel, cbm emitted this error message, with typo:
[ERROR] cbm (src/bootman/update.c:L270): Cannot dermine the currently running kernel
clr-boot-manager/src/bootman/update.c
Line 270 in 9bdd68d
Scenario:
chroot
ing into a new rootfsUEFI
systemESP
in the eltorito alt boot section of an ISO.Problem:
get_boot_device
fails to determine the ESP. Assumes LEGACY
capabilitygrub2
bootloader instead of a systemd-class
bootloaderSolution:
/sys/firmware/efi
to weight the decision by adding the UEFI
capability in bootloader selection./boot
is mounted within the environment and will skip mounting it.Testing:
get_boot_device
, on both MBR and GPT rootfs (see issue #53)On certain systems we're seeing GRUB2 menus default to selecting index 0.
On a system with an existing kernel and an upgrade, an unattended reboot leads to booting the old
kernel.
In terms of usability, for some users they may never end up on the new kernel without explicitly selecting it.
Thus, we should alter GRUB2 to emit a single /etc/grub.d
file, nuke old files, and use submenus for the "alternative boot entries".
This ensures that our default kernel, idx 0, is always selected first, and we stuff older/other entries under a different submenu structure.
Codewise, we'll make the GRUB2 implementation similar to the syslinux implementation.
Kernel build support directories will be created with paths matching /usr/src/linux-$(uname -r), if the linux-dev pundle is installed. They should be removed when the corresponding kernels are removed.
Linking issues when using -Wl,--as-needed
or equivalent.
Resolve immediately
Currently we enum the kernel types which limits future expansion.
Instead, we should simply group by type into an NcHashmap
of NcArray
,
and then enable garbage collection through each series, vs a total set.
From Readme.md
clr-boot-manager is primarily designed for GPT disks using UEFI, however it does contain fallback support for legacy bootloaders such as GRUB2 to allow all users to benefit from automated kernel management.
for me this reads like clr-boot-manager can be the bootloader on GPT disks using UEFI and for legacy (BIOS, MBR, ...) GRUB2 can be configured. But the post on Reddit indicates that clr-boot-manager
is no bootloader itself but is a manager ('orchestration layer') that uses bootloaders
Main ticket items:
machine-id
version
fields, etc)machine-id
into the filenames too$ESP/$bootloader
compatibility paths from tools like bootctl
$ESP/EFI/$NEEDLE/bootloader$SUBARCH.efi
, i.e. \EFI\org.clearlinux\bootloaderx64.efi
register
subcommand for installers to register the bootloaderI apologize if this is the wrong place to post, I figure the solution is in the boot manager ๐ค.
I was wondering if there's a way to modify the monitor resolution for Clear Linux when it's installed on bare metal (and there's no GUI). Would this be possible by modifying /proc/cmdline
?
TL;DR the resolution is too good on my Intel NUC and monitor via HDMI, and I'd like to make the font size a little bit bigger.
TL;DR Get coverage up to 85%
clr-boot-manager
should retain and maintain the UEFI variables (nvvars) for it's own entry, during boot and upgrades. Specifically a policy must be introduced to allow UEFI variables to be configured from an installation process (i.e. swupd
from the Clear Linux installer)
Currently ESP detection is based on the freedesktop bootloader specification.
In order to facilitate #11 we need to have advanced detection for cases in which we didn't use the
same ESP. As such we should enumerate partitions on the same parent disk as the root (/) partition, and find the ESP there. This will emulate the same functionality as get_legacy_boot_device
We've changed the policy to always attempt kernel installation.
To close off this release we need to ensure to only update things when they change, including:
*.conf
and loader.conf
filesRight now we're relying on interleaving of functions for differing operations.
ESP is always a case of /boot != /
, whereas BIOS means /boot ~= /
,
changing the concept of can_mount to should_mount and never_mount.
As such we need to rework set_prefix
to initially inspect devices along the parent tree,
and build a strategy for going forward.
Currently tests for uninstallation only look at the EFI dir, improve tests to account for the rootfs as well.
clr-boot-manager
should be able to rescue a bricked install by rebuilding the ESP content,
and repopulating with the bootloader files, etc.
Copied from clearlinux/distribution#142
I do have several distributions in my system, and all boot through EFI, now ClearLinux thinks, it is the only OS to be in the system and fiddle with the boot section.
Do not write or mess with the boot section...at least ask the end user, that you want to update it or want to do it in a proper way??
Sound naive?? Sound stupid?? can't help. That is what bugs me. I tried to bypass it and it is still bugging.
I am sure, I am certainly missing some arcane details to get on.
Currently (at least on Solus), the old kernel modules are left in /lib/modules
. After a year, the disk usage of this folder is 5.6Gb on my machine.
It would be nice to foresee a mechanism in clr-boot-manager to purge the old kernel modules.
When running an update, if the kernel being installed does not have a corresponding module directory the update is aborted. In some cases modules may not be needed nor present so this behavior limits valid use cases.
clr-boot-manager/man/clr-boot-manager.1.in
Line 101 in 50cc4ea
We should specify that the contents of any cmdline.d/*.conf must be simple text to be appended directly to the kernel command line.
We should also mention that contents from /usr/lib/kernel/cmdline* and /usr/lib/kernel/cmdline.d/*.conf are used in generation of the command line.
Currently we probe for get_legacy_boot_device
first, however..
A report shows that a system with BOTH a legacy boot device, and an ESP, will fail to take the appropriate action for the UEFI booted system.
Proposed change:
!self->image_mode
and host UEFI is available.Would like an installer that can work on IP-based KVMs.
Hi,
I have been using LegacyBIOS with the kvm-kernel and noticed that clr-boot-manager
Perhaps this is fixed in #129 but wanted to file just in case. I assume legacyBIOS is the problem because clr-boot-manager works for me in instances where its not enabled.
For usage outside of the cut-and-dry UEFI + basic rootfs, clr-boot-manager
requires far more support in the core. Additionally, the support required makes the project reusable in other projects too.
os-release
files for OS informationdracut
at present)cmdline
(/etc/kernel/cmdline
and /etc/kernel/cmdline.d/*.conf
)GRUB2
bootloaderThis issue supersedes issue #25
With the update of the clr-boot-manager, the goofiboot is now showing multiple kernel versions as possible boot options. I have ran the clr-boot-manager report-booted and update but the current and previous options remain. I would like to remove the previous option and old kernel data stored but cannot find information on how to do so under the manpages for clr-boot-manager. Suggestions?
Environment: Bare metal installation of Clear
Last working Clear Linux version: 23690
Current Clear Linux version installed: 23950
Trying to set timeout for systemd-boot menu and perform clr-boot-manager update.
$ sudo clr-boot-manager update
[FATAL] cbm: (../src/lib/bootvar.c:L411): efi_generate_file_device_path_from_esp() failed: Function not implemented
[FATAL] cbm: (../src/lib/bootvar.c:L411): efi_generate_file_device_path_from_esp() failed: Function not implemented
[FATAL] cbm: (../src/bootloader/shim-systemd.c:L283): Cannot create EFI variable (boot entry)
[FATAL] cbm: (../src/bootman/update.c:L471): Failed to install bootloader
The timeout is set by system did not receive the update hence the boot menu does not show up when system reboot.
If the system being updated has /boot mounted as a non EFI partition and clr-boot-manager is configured for an EFI bootloader, /boot will still be populated with EFI content.
Likely need to try and find an EFI partition or just not install content. @ikeydoherty Any thoughts on this?
We need to enable tests for every single operation, the tool should hook into library code and set context, it should not be performing the logic itself
syslinux support is to be used to enable MBR/BIOS loading in clr-boot-manager
Subtasks:
Tracking some issues here as I'll need to follow up with some PRs to restore git master functionality.
Currently tests will fail UNLESS --with-bootloader=shim-systemd-boot
. In Travis this passes only because the auto
mode will fallback to shim-systemd-boot
. All other options will fail on check_uefi
.
There is also the issue of files having moved into kernel/
subdirectory, so I'll need to track migration paths.
AR ikey:
--with-bootloader=goofiboot
work again (and for all sd class.)kernel/
tree movingGeneral:
distcheck
is broken on master (William found this) - cmdline tests no longer pass. LIkely related to the above issue.gnu-efi-devel
and efivar-devel
(Solus names but yknow the dudes I mean)Solus requires a solid promise in terms of kernel updates.
..CBM.
Currently we employ two bootloaders, depending on UEFI availability:
In addition, we boot with an initramfs, as Solus has to support mad setups like LVM2/encryption (thankfully handled by the internal initramfs cruft, dracut)
Ideally Solus will be switching to CBM before the next ISO snapshot, making use of the namespace feature in CBM.
N.B: "systemd-class" UEFI, and use of GRUB, are common on many distros in conjunction with an initramfs. It would seem that in implementing this for Solus, I should ensure its relatively agnostic so that others can begin to adopt this too, especially in terms of ESP management.
Enable easier transition through format bump by providing .sh shims
It should be possible to retain settings other than default
and timeout
in the loader.conf
. Specifically, the 4 following options should be retained at all times:
editor
auto-entries
auto-firmware
console-mode
right now these are wiped every time clr-boot-manager runs.
Scenario:
ESP
MBR
disk contains /
Problem:
PartUUID
& UUID
for GPT converted to MBRroot=PARTUUID=
with an unsupported PartUUID
Currently, if we find an ESP, we niavely assume GPT. The probe
code must be modified to determine if the root device is GPT
or not. If it is GPT, we should write the PartUUID, otherwise free and NULL the PartUUID and write only the UUID.
Make another cleanup step in clr-boot-manager which will look at all the names in the target and try to find those files' matching sources. If the matching sources of those files do not exist and the kernel isn't being used (either as default or as a fallback), then cleanup all the target files.
Running clr-boot-manager
in Solus installed on a Btrfs filesystem results in the following:
# CBM_DEBUG=1 clr-boot-manager update
[INFO] cbm (src/bootman/bootman.c:L437): Current running kernel: 4.9.18-10.lts
[ERROR] cbm (src/lib/files.c:L164): Invalid block device: /
[ERROR] cbm (src/lib/system_stub.c:L30): Invalid block device: 0:34
Out of memory[DEBUG] cbm (src/bootman/bootman.c:L124): UEFI boot now selected (goofiboot)
[INFO] cbm (src/bootman/update.c:L75): Checking for mounted boot dir
[INFO] cbm (src/bootman/update.c:L78): boot_dir is already mounted: /boot
[DEBUG] cbm (src/bootman/update.c:L237): Now beginning update_native
[DEBUG] cbm (src/bootman/update.c:L246): update_native: 1 available kernels
[DEBUG] cbm (src/bootman/update.c:L266): update_native: Running kernel is (lts) ///usr/lib/kernel/com.solus-project.lts.4.9.18-10
[FATAL] cbm (src/bootman/sysconfig.c:L126): sysconfig insane: Missing root device
[FATAL] cbm (src/bootman/update.c:L455): Failed to update bootloader
Notes:
/
filesystem is in the top-level subvolume (subvol=/
) or another subvolume (eg: subvol=solus
).0:34
doesn't exist.I recently installed Solus Linux. Their software center has CKB-Next in it. The program itself runs just fine, but for some reason it is not seeing my strafe rgb. I can only see the program settings in which it says: "No devices connected".
Intel is providing the Slim Bootloader as a fast and lightweight open-source reference boot loader. This is commonly used in IOT devices like e.g. infotainment systems in the car.
I would like to see the clr-installer directly supporting the creation of images that are bootable via SBL.
A small refactor will be needed inside the bootman
implementation to enable the detection of either a Legacy BIOS boot partition or the EFI System Partition.
Consequently scope is locked to GPT
disks only.
In Solus we're trying to ensure all aspects of the users system is up to date, to that extent we:
However we currently DO NOT have firmware updates. clr-boot-manager is the obvious integration
point in the deployment chain, and should be able to manage fwupd
or similar to enable the deployment
of FW capsule updates for our UEFI users.
It also makes sense to integrate the points here as CBM is responsible for boot items, and could easily change the "NextBoot" to ensure we reboot to the firmware update screen.
..Discuss? :)
Curating /boot
kernel files requires knowing of potentially 3 kernels:
$kerneldir/default-$type
To track the last known booted kernel we must track this with a kbooted-$short
file to allow mapping this to the kernel availability set, to exclude it from removals.
This can simply be achieved by adding a systemd unit and a tiny binary to run on boot.
Right now updates can seemingly go without issue due to missing required files
Sometimes the running kernel detection can fail, resulting in a chain of errors.
The running kernel should never be removed:
Running kernel: : 4.6.3-238.native
/usr/lib/kernel
post update:
cmdline-4.6.3-235.native config-4.6.3-244.native
cmdline-4.6.3-239.native default-native (-> org.clearlinux.native.4.6.3-244)
cmdline-4.6.3-242.native install.d
cmdline-4.6.3-244.native org.clearlinux.native.4.6.3-235
config-4.6.3-235.native org.clearlinux.native.4.6.3-239
config-4.6.3-239.native org.clearlinux.native.4.6.3-242
config-4.6.3-242.native org.clearlinux.native.4.6.3-244
ESP (curated list showing only CBM specifics)
/boot
|-- EFI
| |-- Boot
| | |-- BOOTX64.EFI
| `-- systemd
| |-- systemd-bootx64.efi
|-- loader
| |-- entries
| | |-- Clear-linux-native-4.6.0-221.conf~
| | |-- Clear-linux-native-4.6.2-232.conf~
| | `-- Clear-linux-native-4.6.3-238.conf
| |-- loader.conf
| `-- loader.conf~
`-- org.clearlinux.native.4.6.3-238
It would appear CBM failed to perform any appropriate actions
Bundle linux-dev pulls in linux-extras, which includes a vmlinux file in /usr/lib/kernel. This file is not cleaned up when an old kernel is removed. It should likely follow the same cleanup logic as System.map-, config-, etc.
The default clear linux boot options has intel_iommu=igfx_off which can't be overridden with intel_iommu=on. Using the standard method of placing a conf file in /etc/kernel/cmdline.d and running clr-boot-manager update to add the intel_iommu=on setting does add the setting but that setting does not override the igfx_off setting. Therefore clr-boot-manager needs a way to remove kernel entries and/or overwrite them.
Relevant: https://solus-project.com/forums/viewtopic.php?f=4&t=7211&start=20#p21727
Just a symlink to the current initrd, just like it exists for default-lts.
Possible?
Thanks.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.