siderolabs / bldr Goto Github PK

Make Pkgfile yaml file.

Allow env: k/v section with list of default variables.

(Add format: specifier?)

Might be useful for graph

Extract common base steps/dependencies into a common LLB node, e.g. in many cases first step is to copy base dependency, we should better extract that as common LLB node.

What is happening right now is that each sub-step tries to copy concurrently hundreds of megabytes of data which basically means our build is more of IO-bound rather than CPU-bound.

I'm being a bit grammar nazi here, sorry for that.

In the readme the first line comment directive for the Pkgfile is called shebang, but technically it is not. Shebang is a sequence of two characters hash sign and exclamation mark https://en.wikipedia.org/wiki/Shebang_(Unix)

I believe that "bang" in shebang comes from the exclamation mark (colloquially called "bang" sometimes).

So technically the first line comment in Pkgfile is just a directive, or magic comment or something like that.

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore: update docker.io/oven/bun docker tag to v1.1.22

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

Skip reduction to sub-tree and display all the paths.

I think it would be useful to allow omitting sources sha cheksum validaiton or at least allow only use either sha256 or sha512, not both.

Take a look at the Makefile in that PR: https://github.com/talos-systems/toolchain/pull/24/files It builds a target twice and compares results, pruning the whole cache between builds. I think it has two problems that should be addressed by bldr:

1. That functionality should be a part of the bldr itself, not in a Makefile.
2. Pruning the whole cache might be too much, but just passing --no-cache is not enough. Maybe bldr could be modified to ignore cache or prune just needed entries.

As we no longer download files directly as part of the build, this should never be required.

In addition to checking checksums, it might be a good idea to check PGP signatures for packages that have them. For example, https://github.com/opencontainers/runc/releases/tag/v1.0.0

Find where it is in the gateway protocol, pick it up and use to set correct env vars

Today we have alpine:3.10 hardcoded in the bldr, probably it's better to make it more flexible and potentially use custom base images.

It should be possible to refer to images (and variants) by their SHA256 digest using the standard syntax NAME[:TAG][@DIGEST]. The correct digest, if present, should be enforced.

It also should be possible to update the digest with bldr update [--dry] if it is already present:

• if an image is specified without a digest (image: autonomy/build-container:latest), then bldr update should do nothing;
• if digest is present (image: autonomy/build-container:latest@sha256:c1a2def75622b7d1d8b4ee508720554269bdb068ba1577bd03a62a0089e9ace3), then bldr update should resolve the tag and update the digest.

   - sources:
-      - url: https://ftp.gnu.org/gnu/make/make-4.3.tar.gz
+      - url: https://ftp.gnu.org/gnu/make/make-4.4.tar.gz
         destination: make.tar.gz
         sha256: e05fdde47c5f7ca45cb697e973894ff4f5d79e13b750ed57d7b66d8defc78e19
         sha512: 9a1185cc468368f4ec06478b1cfa343bf90b5cd7c92c0536567db0315b0ee909af53ecce3d44cfd93dd137dbca1ed13af5713e8663590c4fdd21ea635d78496b

If make-4.3.tar.gz was previously downloaded, that patch does not trigger a rebuild.

Be "smart" here, potentially handling different layouts of download locations (e.g. http file server, GitHub releases, etc.)

Run something like bldr check-versions and receive output similar to:

NAME    VERSION   UPGRADES
make      2.18.1         2.18.2, 2.19.0

Small issues:

• Version can't be extracted from https://github.com/opencontainers/runc/releases/download/v1.0.0-rc95/runc.tar.xz

Prepare some set of pkg.yamls, Pkgfile, test via bldr llb and via bldr frontend (need to push it?)

So that it's easy to spot errors

This makes more sense when we eliminate majority of them, otherwise it would be a huge mess

Custom SHA512 checksum check should be glued into LLB graph so that failures terminates the build

Today we copy empty directory to bind those steps, but this is not clean: https://github.com/talos-systems/bldr/blob/master/internal/pkg/convert/node.go#L158

We could have just make checksummer to be input node for the next LLB step: moby/buildkit#1185

Default variables contain talos many times. If bldr is generic, we should allow this to be overridable. One of the options is to use Pkgfile to provide vendor variable and use that to populate default variables.

Document pkg.yaml, Pkgfile, directory structure, build flow

Document development flow, llb command

Proposal: add runtime flag to stage::

dependencies:
   - stage: m4
   - stage perl:
     runtime: yes

By default, all the dependencies are 'build' dependencies, i.e. they're pulled for the build. If dependency is marked as runtime, then if one stage is pulled in as a dependency, all of its runtime dependencies are pulled in as well.

Examples: automake requires perl (and m4 iirc). So if automake is pulled in as build dependency, it should always pull perl and m4, as automake doesn't run without it.