coreos / coreos-assembler Goto Github PK
View Code? Open in Web Editor NEWTooling container to assemble CoreOS-like systems
Home Page: https://coreos.github.io/coreos-assembler/
License: Apache License 2.0
Tooling container to assemble CoreOS-like systems
Home Page: https://coreos.github.io/coreos-assembler/
License: Apache License 2.0
A local developer workflow probably just involves running coreos-assembler build
over and over again. The problem with this is that build
passes --cache-only
to rpm-ostree compose tree
and thus the rpm repo-metadata won't get refreshed and no new rpms will get pulled.
We should probably think about the design a bit and have stages stack if it makes sense. I keep going back to the rpm-build -ba
thinking. ba
stands for build all or run all stages. bb
is for build binary package (after doing the %prep, %build, and %install stages). bp
executes the %prep stage. bc
executes the %build stage after doing %prep. etc..
I've accumulated quite a few builds in my builds/
dir now.
Let's just keep the last 3 builds around? Any reason to keep less/more?
We currently wrap the architecture name into a board
for historical reasons, and sometimes we treat it as a per-platform option. Cleanups are needed.
There are two main parts of coreos-assembler: the os container and the build scripts. For CL these are actually seperate and the src directory gets bind-mounted into the chroot. Separating them (e.g. into seperate repos) here would have a few advantages:
This came up in the review of #270 and rather than litter that review with extraneous comments, let's move it here.
Shortlist of things to implement:
.format()
, using os.path.join
, etc)pycodestyle
(nee pep8
) across the code basepycodestyle
on each PROther input welcome. ๐
Would be cool to make init
be completely idempotent and only actually error out if it's not purely creating something, but also deleting (in which case we'd require --force
). That'd make pipelines easier as well.
Currently the model is that builds.json
implements a linear stream of builds, a lot like ostree/git commits. But we should add the concept of tagging.
See prior discussion in e.g. #177
What I'm thinking here is that builds.json
would look like:
{ "builds": ["42.4", "42.3", "42.2"]
"tags": [{"name": "smoketested",
"created": "2018-09-26T15:51:42Z",
"target": "42.3"}, ...]
}
Created by e.g.: coreos-assembler build-tag create smoketested 42.3
Our pruning logic would be enhanced to understand these additional references.
We've been abbreviating coreos-assembler
as COSA in conversations more recently. From the beginning of the project I've also been using an alias alias cass=...
to shorten the keystrokes and to allow the alias to handle the complex CLI that needs to be run for the container runtime. Since COSA has taken on some traction recently (and I like it better than cass
) I propose that we formalize the abbreviation of coreo-assembler
to cosa
in some of our docs and within the container itself.
cosa
in README and examples in docscosa -> coreos-assembler
symlink inside of container
All of these points are up for debate, including if we want to bikeshed on the cosa
name.
WDYT?
As per discussion, we should look into moving the mantle bits to their own container. They aren't used to build the image; only for uploading, releasing, testing, etc.
Splitting them out would have the following advantages:
There are of course a few disadvantages:
I keep hitting an issue where I build a FCOS image just fine using coreos-assembler but the resulting image gets MCS labels on the file so when I try to run a second container (i.e. coreos-assembler run
) I get Could not open 'builds/29/fedora-coreos-29-qemu.qcow2': Permission denied
.
here are the labels on the file:
[dustymabe@media srv-coreos]$ ls -lZ builds/29/fedora-coreos-29-qemu.qcow2
-rw-r--r--. 1 1000 1000 system_u:object_r:container_file_t:s0:c255,c505 1905655808 Jan 22 17:07 builds/29/fedora-coreos-29-qemu.qcow2
So I have system_u:object_r:container_file_t:s0:c255,c505
context. I then execute a podman command with a volume mount with :z
for relabeling with sharing contexts and when that is done I get system_u:object_r:container_file_t:s0
context.
[dustymabe@media srv-coreos]$ sudo podman run -it --rm -v $PWD:/foo:z --entrypoint /bin/bash registry.fedoraproject.org/fedora:28
[root@d48949e7d73c /]#
[root@d48949e7d73c /]# exit
exit
[dustymabe@media srv-coreos]$ ls -lZ builds/29/fedora-coreos-29-qemu.qcow2
-rw-r--r--. 1 1000 1000 system_u:object_r:container_file_t:s0 1905655808 Jan 22 17:07 builds/29/fedora-coreos-29-qemu.qcow2
so it removed the MCS labels and now I can coreos-assembler run
.
How can I set the contexts on the directory so that the MCS labels never get applied?
I'm trying to just get through a successful build. My apologies if this is another oops.
The host I'm running on is a Fedora 29 (upgraded from 28 today) VM on ESXi, with virtualization passthrough turned on (/dev/kvm
is available). After doing a clean init, I run coreos-assembler build
per the docs.
The build produces a qcow2 image from virt-install and proceeds to run gf-oemid
, where it fails.
I've attached the full build log.
+ guestfish --remote -- run
libguestfs: error: appliance closed the connection unexpectedly.
This usually means the libguestfs appliance crashed.
filing this to give an update of my findings in getting coreos built on aarch64. The main issues I have hit so far are the calls to qemu-kvm, it is a x86 only binary, along with the grub config being in a different location due to EFI.
I am currently stuck on figuring out how to mount the ESP partition as it does not end up with a disk label.
Not too sure if I should of opened this up here or https://github.com/coreos/fedora-coreos-config
So my scenario is that I've built the Fedora CoreOS image. I've got the .qcow2 image, and now I'm hosting it on a libvirt / KVM box up in the cloud.
The problem is getting SSH access. Obviously OpenSSH is installed and accessible.
Obviously console login won't work: https://github.com/coreos/fedora-coreos-config/blob/master/image.ks#L22
and a user 'core' has been created: https://github.com/coreos/fedora-coreos-config/blob/master/fedora-coreos-base.yaml#L111
Currently coreos-assembler run
will self-inject an SSH key into the image and it's accessible that way.
My question is:
Where do I inject my public SSH key? With the original CoreOS you could provide this in a cloud config / initial config. Should I add this to post-process? https://github.com/coreos/fedora-coreos-config/blob/master/fedora-coreos-base.yaml#L36 or am I missing something (such as an ignition config or kickstart config?)
I was following the directions in the README until it broke with an apparent SELinux issue. I have SELinux enforcing, but I've ran it in permissive mode too with the same error. I haven't tried disabling it altogether, but that seems counter to the Fedora philosophy (and mine).
[devuser@localhost coreos]$ cd /srv/coreos
[devuser@localhost coreos]$ alias coreos-assembler='podman run --rm --net=host -ti --privileged --userns=host -v $(pwd):/srv --workdir /srv quay.io/cgwalters/coreos-assembler'
[devuser@localhost coreos]$ coreos-assembler init https://github.com/coreos/fedora-coreos-config
Trying to pull quay.io/cgwalters/coreos-assembler...Getting image source signatures
Copying blob sha256:e69e955c514f72cef9e9e7db408266a32ba48133f0e2a27d5e4b52ed1995d864
85.99 MB / 85.99 MB [======================================================] 5s
Copying blob sha256:21600e57f2c7e6952d55f945688a9f886d5b2caebece0f4897081760f61ca73f
160 B / 160 B [============================================================] 0s
Copying blob sha256:e398bc963f4ded7eaab2a9bc9e726560df344158d253dc148109af0628ced04a
2.71 KB / 2.71 KB [========================================================] 0s
Copying blob sha256:f64fded7e3a7ecf43bba485a1dfc51f7b9cbdf4f9bf644d01c79fc84d41d4eb3
1.15 KB / 1.15 KB [========================================================] 0s
Copying blob sha256:c11ad1c9e64a591c75dce813c670fc0739bcc4f604c96704e2a9eb30cea29f3a
588.94 MB / 588.94 MB [===================================================] 30s
Copying blob sha256:19a8e73a980698ae96f543930e6e4e76e6bbfa6d0b63616c9f1ce0a5c993dc12
3.51 MB / 3.51 MB [========================================================] 0s
Copying blob sha256:999624bd993360d23122d4753564f8ad21e98fcfb624046eb6d709bb7325340f
169.86 MB / 169.86 MB [===================================================] 11s
Copying blob sha256:efbb6d113281db538c8a453bd9717ccf762f4252de549a8adec7b290fab8be14
3.32 KB / 3.32 KB [========================================================] 0s
Copying blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
32 B / 32 B [==============================================================] 0s
Copying blob sha256:b01f865b2ec506c5b8d510c88bfc7251c54cc16c5fc4502b3b5e79f00d2eeba3
137 B / 137 B [============================================================] 0s
Copying blob sha256:707462d70f0ee977c4e99d1dc2339e1eb7b9508690ea73cc7b442c4813ca90f6
678 B / 678 B [============================================================] 0s
Skipping fetch of repeat blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Skipping fetch of repeat blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Skipping fetch of repeat blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Skipping fetch of repeat blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Writing manifest to image destination
Storing signatures
umount: /sys/fs/selinux: not mounted.
[devuser@localhost coreos]$ getenforce
Enforcing
I've also tried docker, which was able to initialize but I ran into a libguestfs issue there, I'll open separately.
The previous RHCOS build pipeline extracted the pkglist for each build which we found convenient. Let's go the next step and extract the whole rpm-ostree commit metadata as commitmeta.json
or so.
The primary use case here is getting the full package list.
It was suggested that we move the building of the container image to the quay.io registry, but I don't know the repercussions of building the image on a new registry.
What are the benefits and drawbacks to building on Quay? Please discuss!
cc: @lucab @cgwalters @ashcrow
Despite what the README says, this container image does not ship imagefactory
binary in PATH. I've read somewhere else that the image-conversion step is usually performed outside of the container.
As a casual observer, a couple of questions:
imagefactory
to this container, or was it left out on purpose (and why)?As best I can tell...it's like OpenShift builds aren't actually copying the .git
directory. We're doing COPY . .
- this works with both Docker and podman.
Hmm. I wonder if we're hitting openshift/imagebuilder@084da0f
I see this periodically, looks like a race condition:
ERROR Failed to connect socket to '/home/builder/.cache/libvirt/libvirt-sock': No such file or directory
I wonder if it'd help to pre-create /home/builder/.cache
? Also relatedly, we should pre-cache the guestfish VM image with a dry run.
This is the error I see when trying to run this on the ppc64le architecture.
[root@rhel-ocpapp1 coreos]# coreos-assembler init https://github.com/coreos/fedora-coreos-config
Trying to pull quay.io/cgwalters/coreos-assembler...Getting image source signatures
Copying blob sha256:e69e955c514f72cef9e9e7db408266a32ba48133f0e2a27d5e4b52ed1995d864
85.99 MB / 85.99 MB [=====================================================] 10s
Copying blob sha256:a8a4d821e15658a276717ed65d865469e12dabbde7cf1871c755025118b58f3b
160 B / 160 B [============================================================] 0s
Copying blob sha256:99a83b419d3cb4c515d32e0af4ca354f48dd4fdc40872f7bd0e4b939a04afee8
2.53 KB / 2.53 KB [========================================================] 0s
Copying blob sha256:8faffdba319aeddefcd0a7598774221bf6ac3e7ba136bc147e0231a74e6166fc
1.16 KB / 1.16 KB [========================================================] 0s
Copying blob sha256:a417b5e014d6bfb6b81ce057971d82f67739a0c9ebd47dd615939a63255cd7a7
561.86 MB / 561.86 MB [===================================================] 56s
Copying blob sha256:acf8382988a511a2e77d66b6db5d6ab926198625b1d451a21040458b984184ba
3.51 MB / 3.51 MB [========================================================] 0s
Copying blob sha256:b4118eadddddafb8e15b73ac929c6cb0e40a53036d49d87c611982c9c66adf96
169.87 MB / 169.87 MB [===================================================] 19s
Copying blob sha256:134fd40227f49150593822289e2fd1efdf359ec2730aaf6ab01488001eab3b65
3.32 KB / 3.32 KB [========================================================] 0s
Copying blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
32 B / 32 B [==============================================================] 0s
Copying blob sha256:710caf03ad410b4ffd7ae487eb56e51e6a3b58f16a9711f551f03cbed8a40973
137 B / 137 B [============================================================] 0s
Skipping fetch of repeat blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Skipping fetch of repeat blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Skipping fetch of repeat blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Skipping fetch of repeat blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
Writing manifest to image destination
Storing signatures
standard_init_linux.go:190: exec user process caused "exec format error"
Currently, c-a outputs a "traditional" OSTree repository. There is some discussion as to what we do for FCOS.
For Red Hat CoreOS, we are going to be delivering ostree-in-container, or "oscontainer" for short. I would like the assembler to gain a configuration file (more YAML) that allows the builder to specify output formats (we also probably want this for image output types).
Something like:
filename=buildcfg.yml
output: oscontainer|ostree-repo|rojig
images:
- qemu
- openstack
- iso
?
The messy part about oscontainer is dealing with container images inside a container.
Currently the container is built from Fedora userspace, and Fedora hosts have 0666
for /dev/kvm
. Other systems may not.
The easiest fix here is probably to make doubly sure we have our own copy of /dev
(both podman and docker do this I believe), then rm -f /dev/kvm; mknod /dev/kvm && setfacl -m u:$USER:rw /dev/kvm
or so.
We could wrap the existing https://github.com/projectatomic/rpmdistro-gitoverlay that we ship, or do something else (more gentoo like?)
Integrate this so it's still just coreos-assembler build
.
originally reported over in: openshift/os#311 by luca, copied here:
I'm not sure where in the pipeline we are baking in and recording kargs, but it looks rpm-ostree kargs
manipulation drops some of them. In the example below, an --append=foo=bar
results in losing coreos.oem.id=ec2
.
$ ssh core@vm "sudo cat /proc/cmdline"
BOOT_IMAGE=/ostree/rhcos-ea868b22bf773dc384da77e76f66e9c51ae821f86df47d1fdd9ce62d0bebb04c/vmlinuz-3.10.0-953.el7.x86_64
no_timer_check console=tty0 console=ttyS0,115200n8 net.ifnames=0 biosdevname=0 rd.neednet=1 rw coreos.firstboot=detected ip=eth0:dhcp
root=UUID=2efac7b2-3518-43ca-9e90-40649c28f7a0
ostree=/ostree/boot.0/rhcos/ea868b22bf773dc384da77e76f66e9c51ae821f86df47d1fdd9ce62d0bebb04c/0
coreos.oem.id=ec2
$ ssh core@vm "sudo rpm-ostree kargs --reboot --append=foo=bar"
$ ssh core@vm "sudo cat /proc/cmdline"
BOOT_IMAGE=/ostree/rhcos-ea868b22bf773dc384da77e76f66e9c51ae821f86df47d1fdd9ce62d0bebb04c/vmlinuz-3.10.0-953.el7.x86_64
no_timer_check console=tty0 console=ttyS0,115200n8 net.ifnames=0 biosdevname=0 rd.neednet=1 rw ip=eth0:dhcp
root=UUID=2efac7b2-3518-43ca-9e90-40649c28f7a0
ostree=/ostree/boot.1/rhcos/ea868b22bf773dc384da77e76f66e9c51ae821f86df47d1fdd9ce62d0bebb04c/0
foo=bar
gf-oemid should also modify the BLS config file
I've been thinking about #184
Right now the data model is pretty simple. If we add tagging it gets more complex.
One thing I've been thinking about, and this is inspired by previous discussions around rpm-md - what if we took a page from Rust's cargo and stored our builds.json
as well as the meta.json
in git?
Then our tags would literally just be git tag
.
Another approach is to store builds in ostree: #189 (comment)
(However today ostree doesn't have tags as such)
The main reason I want to do this is to retain some history of builds even after we do some pruning in production.
I was testing changes to the input repos on an RHCOS build and tried to switch git
repos using init --force
, but found that init
didn't try to pull the new repo.
I know that --force
is there to ignore non-empty directories, but should it's meaning be extended to include more than that?
If not, would another flag be useful for this kind of "big hammer" operation where everything in src
is replaced?
The fine folks over at http://github.com/coreos/ignition have a JSON schema for the Ignition configs that doubles as the spec for the project.
https://github.com/coreos/ignition/blob/master/schema/ignition.json
While I don't think have a need for a spec for this project, having schemas defined for the JSON files that are generated will help us check for regressions as we make changes to the project.
Hi, IMHO one of the most exciting features of podman is the rootless execution of containers.
I have tried to share /dev/kvm
using --mount=type=bind,source=/dev/kvm,destination=/dev/kvm
.
This works for init
and fetch
, but build
fails:
user@host ~/coreos % podman --log-level info run -ti --rm --net=host --userns=host --privileged -v $(pwd):/srv --workdir /srv --mount=type=bind,source=/dev/kvm,destination=/dev/kvm quay.io/coreos-assembler/coreos-assembler build
INFO[0000] running as rootless
INFO[0000] Found CNI network podman (type=bridge) at /etc/cni/net.d/87-podman-bridge.conflist
INFO[0000] running as rootless
INFO[0000] Found CNI network podman (type=bridge) at /etc/cni/net.d/87-podman-bridge.conflist
WARN[0037] Failed to add conmon to cgroupfs sandbox cgroup: mkdir /sys/fs/cgroup/systemd/libpod_parent: permission denied
Using manifest: /srv/src/config/manifest.yaml
libostree:
Version: '2018.10'
Git: c5aaabe9d754d2ada17d5cc006e13105ab5cecc2
DevelBuild: yes
Features:
- libcurl
- no-http2
- libsoup
- gpgme
- libarchive
- selinux
- openssl
- libmount
- devel
- p2p
rpm-ostree:
Version: '2018.9'
Git: 0e24944c4ec0a45eb88169a60c96c8b76e86cd61
Features:
- compose
- rust
Previous commit: none
Kickstart checksum: 93de3934c4488ab094a1b8ffce1b00ac1cf8d1c4d7091519202ceb484f2d25fc
Running: rpm-ostree compose tree --repo=/srv/repo --cachedir=/srv/cache --touch-if-changed /srv/tmp/treecompose.changed --unified-core /srv/src/config/manifest.yaml --cache-only --add-metadata-from-json /srv/tmp/build/tmp/commit-metadata-input.json --write-composejson-to /srv/tmp/build/tmp/compose.json
RPM-OSTree Version: 2018.9
No previous commit for fedora/29/x86_64/coreos
Enabled rpm-md repositories: dustymabe-ignition fedora fedora-updates fedora-updates-testing
rpm-md repo 'dustymabe-ignition' (cached); generated: 2018-11-21T22:40:42Z
rpm-md repo 'fedora' (cached); generated: 2018-10-24T22:20:15Z
rpm-md repo 'fedora-updates' (cached); generated: 2018-11-23T02:15:41Z
rpm-md repo 'fedora-updates-testing' (cached); generated: 2018-11-23T02:31:21Z
Importing rpm-md... done
Resolving dependencies... done
Installing 388 packages:
GeoIP-1.6.12-4.fc29.x86_64 (fedora)
...
zlib-1.2.11-14.fc29.x86_64 (fedora)
Input state hash: 22c7dab4135b85f9d8bea7c3d4e75989a1460b566e1c60f2f5a6972d6f700cdc
Checking out packages... done
โ Running pre scripts... libini_config
fuse: device not found, try 'modprobe fuse' first
โ Running pre scripts... dbus-common
dbus-common.prein: bwrap: execvp /bin/sh: No such file or directory
fusermount: failed to unmount /tmp/rpmostree-rofiles-fuse.XODVvm: Invalid argument
Running pre scripts... done
error: Running %prein for dbus-common: Executing bwrap(/bin/sh): Child process killed by signal 1
podman --log-level info run -ti --rm --net=host --userns=host --privileged -v 11,15s user 21,99s system 54% cpu 1:00,25 total
Do you see any chance to enable rootless builds?
podman version 0.11.1.1
Arch Linux
now that f29 is out it should be safe to switch to it for our assembler container
I've been developing a proof of concept bare metal installer piggybacking Anaconda kickstart functionality.
It doesn't use almost any of the "core" Anaconda features, but it seems to do what it has to do.
Please have a look, @cgwalters and others.
#Kickstart
cmdline
lang en_US.UTF-8
keyboard us
timezone --utc America/New_York
####################################################
# Not used! Fool anaconda into thinking we actually
# set a root password.
rootpw arandomone
#####################################################
####################################################
# This is again kind of useless, as we are going to
# dd the sys image anyway later.
####################################################
clearpart --all --drives=notexistent
autopart --nolvm
network --onboot yes --device ens3 --bootproto dhcp --noipv6
%pre --erroronfail --interpreter /bin/bash --log /tmp/ks.log
#!/bin/bash
##########################################################
# Defaults - edit these to tune the installation
##########################################################
IMG_SERVER_ADDRESS=192.168.1.xxx
DEST_DEV=/dev/sda
IMAGE_URL="http://${IMG_SERVER_ADDRESS}/os-img.img.gz"
#SIG_URL="http://${IMG_SERVER_ADDRESS}/os-img.img.gz.sha256sum"
###########################################################
# Hacks galore - kill shell window and show (our) logs
###########################################################
tmux kill-window -t anaconda:shell
tmux kill-window -t anaconda:log
touch /tmp/install.log
tmux new-window -d -n log "tail -F /tmp/install.log"
tmux select-window -t anaconda:log
echo "Installing System Image" >> /tmp/install.log
echo "-----------------------------------" >> /tmp/install.log
############################################################
#Get the image info
############################################################
let retry=0
while true
do
echo "Getting image info from $IMAGE_URL" >> /tmp/install.log
curl -sIf $IMAGE_URL >/tmp/image_info 2>&1
RETCODE=$?
if [ $RETCODE -ne 0 ]
then
if [ $RETCODE -eq 22 -a $retry -lt 5 ]
then
# Network isn't up yet, sleep for a sec and retry
sleep 1
let retry=$retry+1
continue
fi
exit 1
else
IMAGE_SIZE=$(cat /tmp/image_info | awk '/.*Content-Length.*/ {print $2}' | tr -d $'\r')
TMPFS_MBSIZE=$(($IMAGE_SIZE/(1024*1024)+50))
echo "Image size is $IMAGE_SIZE" >> /tmp/debug
echo "tmpfs sized to $TMPFS_MBSIZE MB" >> /tmp/debug
break;
fi
rm -f /tmp/image_url
done
#########################################################
#Create the tmpfs filesystem to store the image
#########################################################
echo "Mounting tmpfs" >> /tmp/debug
mkdir -p /mnt/dl
mount -t tmpfs -o size=${TMPFS_MBSIZE}m tmpfs /mnt/dl
############################################################
#Figure out the signature file type
############################################################
echo "Getting SIG_URL $SIG_URL" >> /tmp/debug
curl -sIf $SIG_URL > /dev/null 2>&1
RETCODE=$?
if [ $RETCODE -ne 0 ]
then
echo "$SIG_URL not found" >> /tmp/debug
SIG_URL=$IMAGE_URL.sha256sum
echo "Getting SIG_URL $SIG_URL" >> /tmp/debug
curl -sI $SIG_URL > /dev/null 2>&1
if [ $? -ne 0 ]
then
SIG_TYPE=none
else
SIG_TYPE=sha
fi
else
SIG_TYPE=gpg
fi
echo "SIGNATURE TYPE IS $SIG_TYPE" >> /tmp/debug
#########################################################
#And Get the Image
#########################################################
echo "Downloading install image" >> /tmp/install.log
curl -s -o /mnt/dl/imagefile.gz $IMAGE_URL
if [ $? -ne 0 ]
then
exit 1
fi
#########################################################
#Get the corresponding signature file
#########################################################
echo "Getting signature $SIG_URL" >> /tmp/install.log
curl -s -o /mnt/dl/imagefile.gz.sig $SIG_URL
if [ $? -ne 0 ]
then
exit 1
fi
#########################################################
#Validate the integrity of the image
#########################################################
if [ "$SIG_TYPE" != "none" ]
then
if [ "$SIG_TYPE" == "gpg" ]
then
gpg --trusted-key "${GPG_LONG_ID}" --verify /mnt/dl/imagefile.gz.sig >/dev/null 2>&1
if [ $? -ne 0 ]
then
exit 1
fi
elif [ "$SIG_TYPE" == "sha" ]
then
sed -i -e"s/$/\ \/mnt\/dl\/imagefile\.gz/" /mnt/dl/imagefile.gz.sig
sha256sum -c /mnt/dl/imagefile.gz.sig
if [ $? -ne 0 ]
then
exit 1
fi
else
sleep 3
fi
else
sleep 3
fi
#########################################################
#Wipe any remaining disk labels
#########################################################
dd conv=nocreat count=1024 if=/dev/zero of="${DEST_DEV}" \
seek=$(($(blockdev --getsz "${DEST_DEV}") - 1024)) status=none
#########################################################
#And Write the image to disk
#########################################################
echo "Writing disk image" >> /tmp/install.log
gzip -dc /mnt/dl/imagefile.gz | dd conv=nocreat of="${DEST_DEV}" bs=1M status=none
udevadm settle
echo "Install complete. Rebooting in 3 seconds..." >> /tmp/install.log
sleep 3
shutdown -r now
%end
Let's have a default model where a developer or test process can do:
# mkdir -p local-overrides
# cp /path/to/kernel.rpm local-overrides
And if we find any RPMs in there we automatically run createrepo_c
and add it to the config. One thing this will then hit is: rpm-software-management/libdnf#149
Which...hacking around would be very painful.
We have an outstanding discussion about using the plume
bits from mantle, or using the /usr/bin/aws
CLI, etc.
I personally find shell-scripting aws
to not be too bad, see e.g.: openshift/os#330
The mantle* artifacts are the only ones we don't pull from somewhere else. Building them as part of the container build means that we have to download build deps, compile, remove build deps, etc..
It would be better if we could grab the mantle* software pre-built rather than have to do this.
note this mostly came up when discussing how to better leverage podman build --layers
for layer caching while hacking on the coreos-assembler
container sources.
The ostree
and rpm-ostree
projects have a robust set of tests that could be used to validate functionality on RHCOS/FCOS. Specifically, the vmcheck tests of rpm-ostree
are such that they can be run against a VM which has an rpm-ostree
daemon running.
While experimenting running these tests against RHCOS (without using kola
), I ran into a few snags that made me question how the integration of the tests into kola
might work.
The upstream tests can change with each release/commit, so kola
would have to perform a checkout of the repo which matches the version of software under test. I didn't find any other examples of this kind of approach in the current codebase. Obviously, this doesn't mean we can't adapt, but I wanted to get a feel if this kind of approach would work out.
I found the vmcheck
suite has a decent amount of dependencies that need to be installed on the host running the test. kola
doesn't really have this problem, as it is just a golang binary after it is all compiled etc. Would this requirement on certain packages get pushed up to a higher layer to handle?
The alternative to installing packages on the host running the test would be to pull a container with the required dependencies and execute the tests via the container, via kola
. I'm not sure this is a better solution than what would be required for point 2.
The extreme alternative is to punt on the upstream tests and re-implement some of the tests natively in 'kola`.
Maybe I'm getting ahead of myself with these kinds of questions, but I figured the folks more experienced with this codebase would be able to steer me in the right direction.
During the coreos-assembler build
process on a Fedora 29 VM with nested kvm enabled, I'm getting a ton of cp: setting attributes for XXX: Operation not supported
errors before the operation errors out. This is with podman, but with docker the operation goes further (It's stopping much later on, with a ModuleNotFoundError: No module named 'pytz'
error -- I'll follow up on that separately).
dracut: FAILED: /usr/lib/dracut/dracut-install -D /tmp/dracut/dracut.HD3Ged/initramfs /usr/lib/dracut/modules.d/30ignition/ignition-files.service /usr/lib/systemd/system/ignition-files.service
cp: setting attributes for '/tmp/dracut/dracut.HD3Ged/initramfs/usr/lib/systemd/system/ignition-ask-var-mount.service': Operation not supported
dracut-install: ERROR: installing '/usr/lib/dracut/modules.d/30ignition/ignition-ask-var-mount.service' to '/usr/lib/systemd/system/ignition-ask-var-mount.service'
dracut: FAILED: /usr/lib/dracut/dracut-install -D /tmp/dracut/dracut.HD3Ged/initramfs /usr/lib/dracut/modules.d/30ignition/ignition-ask-var-mount.service /usr/lib/systemd/system/ignition-ask-var-mount.service
cp: setting attributes for '/tmp/dracut/dracut.HD3Ged/initramfs/lib/modules/4.19.8-300.fc29.x86_64//kernel/drivers/firmware/qemu_fw_cfg.ko.xz': Operation not supported
dracut-install: dracut_install 'qemu_fw_cfg' ERROR
dracut-install: ERROR: installing 'qemu_fw_cfg'
dracut: FAILED: /usr/lib/dracut/dracut-install -D /tmp/dracut/dracut.HD3Ged/initramfs --kerneldir /lib/modules/4.19.8-300.fc29.x86_64/ -m qemu_fw_cfg
dracut: installkernel failed in module ignition
error: Finalizing rootfs: During kernel processing: Executing bwrap(rpmostree-dracut-wrapper): Child process killed by signal 1
I've been developing a proof of concept Bare metal installer making use of the dracut initrd building system, by creating a dracut module specifically created to run a modified version of the coreos-installer script (available here https://github.com/nhorman/coreos-dracut ). Compared to the @jlelli POC here (#228), this solution makes no use of anaconda, and is therefore smaller in size and complexity, but also is unable to take advantage of any of the features offered in anaconda, should that be desireable in the future.
to build an initrd that is capable of installing coreos:
Using either the pxe or cdrom method of boot, makes the following kernel command line options available:
coreos.install_dev - The block device on the system to install to
coreos.image_url - The url of the coreos image to install to this device
coreos.ignition_url - The url of the coreos ignition config (optional, enter coreos.ignition_url=skip to not load an ignition config)
If the install_dev and image_url options are specified, the installer will run autonomously. If they are not specified, the installer will become interactive, prompting the user for input on the console
Basically rpm-ostree doesn't write the --write-composejson-to
if nothing changed. If it did, this would be fixed.
In this comment #214 (comment), @wking appears to be asking for timestamps associated with the builds in builds.json
Before I go any deeper into this hole (I've already got some rough changes made), does this makes sense to implement now? And, more importantly, did I understand the ask correctly?
What I've got so far produces a JSON that now looks like this:
{
"builds": [
{
"29.12": {
"timestamp": "2018-11-14 16:48:56+00:00"
}
},
{
"29.11": {
"timestamp": "2018-11-13 16:10:58+00:00"
}
},
{
"29.9": {
"timestamp": "2018-11-06 04:38:21+00:00"
}
}
]
}
WDYT?
Today c-a defaults to handling "local development". I am working on orchestrating it via Jenkins and storing builds in S3. This issue is about the design of the latter.
Currently I have a script which pulls the builds.json
and then fetches the meta.json
from the previous build - basically setting up enough of a "skeleton" that c-a can start a new build.
One approach we could take is to offer this as a subcommand; something like storage-s3-pull
or so?
Basically, we don't try to directly integrate this into a high level flow, just make it something that can be invoked by an orchestrating pipeline.
On a related topic, the Red Hat CoreOS builds use the oscontainer
subcommand in this way right now.
The tool tells you to use --force-nocache
to ignore the cache while building, but this appears not to work.
No apparent changes since previous commit; use --force-nocache to override
No changes in image inputs.
[root@fedora-1 coreos]# coreos-assembler build --force-nocache
getopt: unrecognized option '--force-nocache'
Usage: coreos-assembler build --help
coreos-assembler build [--force] [--skip-prune]
Build OSTree and image artifacts from previously fetched packages.
[root@fedora-1 coreos]# coreos-assembler build --force
Should we disallow init
unless we are in an empty directory? Several times I've run init when I was in the wrong directory and it has littered files in places I didn't want them. We could
init
more than once is desirable then we can just place a file in the SDK directory that let's us know this is an SDK directory.We should probably make all 'init' functions idempotent so it can be run multiple times. Ansible comes to mind.
For coreos-assembler run
it would be nice if we can print a message to the console to tell the user how to break out of the machine. I think once https://github.com/rfairley/console-login-helper-messages lands (or maybe just the pam upstream stuff is needed) we should be able to do this easily.
See also: #49 (comment) and
https://superuser.com/questions/1087859/how-to-quit-the-qemu-monitor-when-not-using-a-gui
While exploring options for implementing a more complex RHCOS version string (openshift/os#360), I thought about extending the buildprep
stage to be able to run scripts as part of the stage. These could be thought of like "plugins" to the buildprep
stage.
I envisioned a buildprep
directory (maybe it should be plugins
?) that would live alongside the the manifest.yaml
for a project. Inside the directory could be executable scripts that would perform additional actions to prep the directory for building.
For example, in RHCOS we need to do additional steps to generate repo files before we can do an actual build. Instead of requiring the build system or user to do those steps, buildprep
could read the scripts in the directory and do those steps as part of the stage.
I think this idea would also change how build
would operate, in that we would probably want to run buildprep
implicitly when doing build
.
Does this idea seem useful?
This follows up to coreos/fedora-coreos-config#32 (comment) (list item 3).
console-login-helper-messages
generates a motd at /run/motd.d/console-login-helper-messages.motd
(see second box of output at coreos/fedora-coreos-config#32 (comment)), from this script. This motd contains information sourced from /etc/os-release
, which CL has done.
It'd be good to have the generated motd displayed when logging in during coreos-assembler run
, so that information can be sourced from /etc/os-release
at runtime (e.g. ANSI_COLOR for the OS release, the OS version, anything else from other system files...).
Showing the generated motd would mean the static line at https://github.com/coreos/fedora-coreos-config/blob/master/fedora-coreos-base.yaml#L129 can be removed (but still keep the rest of the static information).
In order to show the generated motd
, login
can be configured to show a list of motds with the path to the generated motd given in addition to /etc/motd
(see coreos/fedora-coreos-config#32 (comment) (list item 3)).
It'd be a later decision to carry the changes in the /etc/login.defs
config forward for an FCOS release. For now, something like using Ignition to write the /etc/login.defs
with the desired config can be done in the cmd-run
script only.
When I am running a coreos-assembler
container from quay.io, the only identifier I have is the image ID.
I should be able to do podman inspect
and get this information.
Maybe even a coreos-assembler version
command?
I'm looking at implementing the GPG signing of artifacts from coreos-assembler
as requested in openshift/os#362. (I'll start with the metadata files, but extending to all artifacts should be possible. Before writing a bunch of code in the wrong direction, I wanted to get feedback from folks.
Should this kind of functionality be a separate command? For example:
$ coreos-assembler sign --help
Usage: coreos-assembler sign --help
coreos-assembler sign [--all] [--metadata] [--images] GPGKEY
Sign artifacts with GPGKEY. Defaults to signing all artifacts. Can sign only
metadata files or image files.
Or maybe give each stage that produces a --sign
option? For example:
$ coreos-assembler build --help
Usage: coreos-assembler build --help
coreos-assembler build [--force] [--skip-prune] [--sign] [--gpgkey GPGKEY]
Build OSTree and image artifacts from previously fetched packages. May optionally
GPG sign metadata files and images using --sign and providing a GPGKEY.
My preference would be the former option, as it would appear to be the last thing you would want to do before publishing any of the artifacts generated by coreos-assembler
. And it would likely be easier to implement in one place, rather than all over the place.
Thoughts?
We started talking about this in #84, and it seems like we should decide on this soon-ish since the assembler is constantly growing and the longer we wait, the harder the rewrite will be.
To summarize, it seemed like the two likely candidates left were Python and Go. One point brought up in favour of Python was verbose error handling in Go, and another in favour of Go was static type checking.
Add an option to the build command to output an initramfs.img and coreos.img to serve over PXE. This would support the https://github.com/coreos/matchbox project which would lead to tectonic integration. I'd be more than willing to contribute to this.
EDIT: I see a lot of parallels with https://github.com/weldr/lorax/blob/master/docs/livemedia-creator.rst
For RHCOS prototyping out with this so far...caching the installer ISO isn't hard, but is going to complicate the build flow.
Given the likely future direction I'd like to cache the installer ISO in the container image as well, so that it's trivial to mirror everything. Will do a PR if no one objects.
I think we should have this container come opinionated out of the box with everything needed to generate a Fedora CoreOS like system. We shouldn't include the installer ISO or RPMs - but we should have a single command that makes it easy to mirror those, and after that the system can run "offline".
Now, the next big topic: anyone who wants to make a custom thing derives from the Fedora CoreOS defaults (by default).
This means that e.g. the equivalent of https://pagure.io/fedora-atomic should get "baked" into this container. (It doesn't mean that the config needs to live in this repo itself - but we should bake it in)
We should add a new toplevel config file that makes it trivial to override a few things:
And on the last point, I'd also like us to be opinionated about one public cloud way (e.g. S3) to publish content out of the gate, and also support "plain directory" (NFS/whatever).
Major bonus: We make this container trivial to run on a Fedora CoreOS VM in GCE with nested virt enabled.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.