coreos / airlock Goto Github PK

Airlock uses node UUID to identify HTTP clients and lock holders. However, when serialized to textual form, UUID may have several representations. In particular, there are two that are relevant here:

• canonical form: 28c20a4a-64dd-488d-925c-1d591637ca8e
• systemd form: 28c20a4a64dd488d925c1d591637ca8e

Dashes are just visual sugar for humans. Systemd uses the non-dashed form everywhere by default.
We should pick one of the two forms, document it, and use it everywhere in an uniform way.

Airlock should count the number of error responses, on both API endpoints separately. Those two counters should be then exposed as Prometheus metrics via the status service.

After #17, each API has an associated machine-friendly "kind" (with bounded and low cardinality). Error kind can be used as a metric label to enhance observability.

Overall, final metrics should look like the following:

• airlock_v1_pre_reboot_response_errors_total{kind="<XYZ>"}
• airlock_v1_steady_state_response_errors_total{kind="<XYZ>"}

Airlock needs to have additional knobs to configure a "maintenance window" for reboots. Those needs to added both for the default group, and for specific custom groups.

A reboot window works in conjunction (logical AND) with existing counting-semaphore logic.

Entries to be added to the lock section:

• default_reboot_window_days: a set of short-day-names (like date %a, i.e. [ "Mon", "Wed" ])
• default_reboot_window_start: start time, in 24h format (i.e. 23:45)
• default_reboot_window_duration: length, in minutes

Entries to be added for each lock.groups section (these override defaults, if set):

• reboot_window_days
• reboot_window_start
• reboot_window_duration

Note: this relies on the system clock of the environment where an airlock process is running.

For operational maintenance and introspection, airlock should grow some CLI subcommands.

Those are mostly meant for human consumption, in order to inspect state in etcd3 and act on it.

The idea is that the administrator can kubectl exec into the container and run maintenance commands there. Those commands requires access to the configuration, which is already available inside the container.

Here below is an initial list of tasks I'd like to be able to perform via CLI.

• lock a reboot slot (by group+uuid)
• unlock a reboot slot (by group+uuid)
• show current slots usage (optionally, filtered by group)
• list configured groups and their configured slots (optionally, filtered by group)

The arm64 container currently builds in an emulated arm64 host via qemu-user. It would be much more efficient to cross-build from amd64 by having the Dockerfile specify FROM --platform=$BUILDPLATFORM for the builder container and set GOARCH=$TARGETARCH. However, Buildah < 1.24.1 doesn't support --platform in FROM. Once a new enough Buildah has landed in ubuntu-latest, switch to cross-building, and re-enable arm64 container builds in PRs by dropping the pr-arches override.

Followup to #31. See also coreos/butane#334.

Capturing the discussion at https://github.com/coreos/airlock/pull/5/files#r279369060.

Manager.UnlockIfHeld can avoid a write-trip if the current id is not holding any semaphore lock. We should first augment Semaphore.UnlockIfHeld to signal back such situation.

This is a braindumping and tracking ticket for selecting and then exposing more configuration options via TOML. I'm still not settled on what's needed, so I'm collecting everything here. Feel free to chime in with more requests/suggestions.

• service
  • path_prefix (common API namespace)
  • misc TLS knobs (specific entries TBD)
• etcd3
  • username (client auth)
  • password_file (client auth)
  • dial_timeout_ms (transport timeout)
  • tls_cert_file (TLS auth)
  • tls_key_file (TLS auth)
• status (metrics exposition and more)
  • address (HTTP socket)
  • port (HTTP socket)