Code Monkey home page Code Monkey logo

uds-cli's Introduction

UDS-CLI

Latest Release Go version Build Status OpenSSF Scorecard

โš ๏ธ Warning: UDS-CLI is in early alpha, expect changes to the schema and workflow

Table of Contents

  1. Install
  2. Contributing
  3. Quickstart
  4. Bundle Architecture and Multi-Arch Support
  5. Configuration
  6. Sharing Variables
  7. Duplicate Packages and Naming
  8. Zarf Integration
  9. Bundle Overrides
  10. Bundle Anatomy
  11. Runner
  12. Dev Mode

Install

Recommended installation method is with Brew:

brew tap defenseunicorns/tap && brew install uds

UDS CLI Binaries are also included with each Github Release

Contributing

Build instructions and contributing docs are located in CONTRIBUTING.md.

Quickstart

The UDS-CLI's flagship feature is deploying multiple, independent Zarf packages. To create a UDSBundle of Zarf packages, create a uds-bundle.yaml file like so:

kind: UDSBundle
metadata:
  name: example
  description: an example UDS bundle
  version: 0.0.1

packages:
  - name: init
    repository: ghcr.io/defenseunicorns/packages/init
    ref: v0.33.0
    optionalComponents:
      - git-server
  - name: podinfo
    repository: ghcr.io/defenseunicorns/uds-cli/podinfo
    ref: 0.0.1

The above UDSBundle deploys the Zarf init package and podinfo.

The packages referenced in packages can exist either locally or in an OCI registry. See here for an example that deploys both local and remote Zarf packages. More UDSBundle examples can be found in the src/test/bundles folder.

Declarative Syntax

The syntax of a uds-bundle.yaml is entirely declarative. As a result, the UDS CLI will not prompt users to deploy optional components in a Zarf package. If you want to deploy an optional Zarf component, it must be specified in the optionalComponents key of a particular package.

First-class UDS Support

When running deploy,inspect,remove, and pull commands, UDS CLI contains shorthand for interacting with the Defense Unicorns org on GHCR. Specifically, unless otherwise specified, paths will automatically be expanded to the Defense Unicorns org on GHCR. For example:

  • uds deploy unicorn-bundle:v0.1.0 is equivalent to uds deploy ghcr.io/defenseunicorns/packages/uds/bundles/unicorn-bundle:v0.1.0

The bundle matching and expansion is ordered as follows:

  1. Local with a tar.zst extension
  2. Remote path: oci://ghcr.io/defenseunicorns/packages/uds/bundles/<path>
  3. Remote path: oci://ghcr.io/defenseunicorns/packages/delivery/<path>
  4. Remote path: oci://ghcr.io/defenseunicorns/packages/<path>

That is to say, if the bundle is not local, UDS CLI will check path 2, path 3, etc for the remote bundle artifact. This behavior can be overriden by specifying the full path to the bundle artifact, for example uds deploy ghcr.io/defenseunicorns/dev/path/dev-bundle:v0.1.0.

Bundle Create

Pulls the Zarf packages from the registry and bundles them into an OCI artifact.

There are 2 ways to create Bundles:

  1. Inside an OCI registry: uds create <dir> -o ghcr.io/defenseunicorns/dev
  2. Locally on your filesystem: uds create <dir>

Note

The --insecure flag is necessary when interacting with a local registry, but not from secure, remote registries such as GHCR.

Bundle Deploy

Deploys the bundle

There are 2 ways to deploy Bundles:

  1. From an OCI registry: uds deploy ghcr.io/defenseunicorns/dev/<name>:<tag>
  2. From your local filesystem: uds deploy uds-bundle-<name>.tar.zst

Specifying Packages using --packages

By default all the packages in the bundle are deployed, but you can also deploy only certain packages in the bundle by using the --packages flag.

As an example: uds deploy uds-bundle-<name>.tar.zst --packages init,nginx

Resuming Bundle Deploys using --resume

By default all the packages in the bundle are deployed, regardless of if they have already been deployed, but you can also choose to only deploy packages that have not already been deployed by using the --resume flag

As an example: uds deploy uds-bundle-<name>.tar.zst --resume

Bundle Inspect

Inspect the uds-bundle.yaml of a bundle

  1. From an OCI registry: uds inspect oci://ghcr.io/defenseunicorns/dev/<name>:<tag>
  2. From your local filesystem: uds inspect uds-bundle-<name>.tar.zst

Viewing SBOMs

There are 2 additional flags for the uds inspect command you can use to extract and view SBOMs:

  • Output the SBOMs as a tar file: uds inspect ... --sbom
  • Output SBOMs into a directory as files: uds inspect ... --sbom --extract

This functionality will use the sboms.tar of the underlying Zarf packages to create new a bundle-sboms.tar artifact containing all SBOMs from the Zarf packages in the bundle.

Bundle Publish

Local bundles can be published to an OCI registry like so: uds publish <bundle>.tar.zst oci://<registry>

As an example: uds publish uds-bundle-example-arm64-0.0.1.tar.zst oci://ghcr.io/github_user

Bundle Remove

Removes the bundle

There are 2 ways to remove Bundles:

  1. From an OCI registry: uds remove oci://ghcr.io/defenseunicorns/dev/<name>:<tag> --confirm
  2. From your local filesystem: uds remove uds-bundle-<name>.tar.zst --confirm

By default all the packages in the bundle are removed, but you can also remove only certain packages in the bundle by using the --packages flag.

As an example: uds remove uds-bundle-<name>.tar.zst --packages init,nginx

Logs

Note

Only works with uds deploy for now, may work for other operations but isn't guaranteed.

The uds logs command can be used to view the most recent logs of a bundle operation. Note that depending on your OS temporary directory and file settings, recent logs are purged after a certain amount of time, so this command may return an error if the logs are no longer available.

Bundle Architecture and Multi-Arch Support

There are several ways to specify the architecture of a bundle:

  1. Setting --architecture or -a flag during uds ... operations: uds create <dir> --architecture arm64
  2. Setting the metadata.architecture key in a uds-bundle.yaml
  3. Setting a UDS_ARCHITECTURE environment variable
  4. Setting the options.architecture key in a uds-config.yaml

Note that the setting the --architecture flag takes precedence over all other methods of specifying the architecture.

UDS CLI supports multi-arch bundles. This means you can push bundles with different architectures to the same remote OCI repository, at the same tag. For example, you can push both an amd64 and arm64 bundle to ghcr.io/<org>/<bundle name>:0.0.1.

Configuration

The UDS CLI can be configured with a uds-config.yaml file. This file can be placed in the current working directory or specified with an environment variable called UDS_CONFIG. The basic structure of the uds-config.yaml is as follows:

options:
   log_level: debug
   architecture: arm64
   no_log_file: false
   no_progress: false
   uds_cache: /tmp/uds-cache
   tmp_dir: /tmp/tmp_dir
   insecure: false
   oci_concurrency: 3

shared:
   domain: uds.dev # shared across all packages in a bundle

variables:
  my-zarf-package:  # name of Zarf package
    ui_color: green # key is not case sensitive and refers to name of Zarf variable
    UI_MSG: "Hello Unicorn"
    hosts:          # variables can be complex types such as lists and maps
       - host: burning.boats
         paths:
            - path: "/"
              pathType: "Prefix"

The options key contains UDS CLI options that are not specific to a particular Zarf package. The variables key contains variables that are specific to a particular Zarf package. If you want to share insensitive variables across multiple Zarf packages, you can use the shared key, where the key is the variable name and the value is the variable value.

Sharing Variables

Importing/Exporting Variables

Zarf package variables can be passed between Zarf packages:

kind: UDSBundle
metadata:
  name: simple-vars
  description: show how vars work
  version: 0.0.1

packages:
  - name: output-var
    repository: localhost:888/output-var
    ref: 0.0.1
    exports:
      - name: OUTPUT
  - name: receive-var
    repository: localhost:888/receive-var
    ref: 0.0.1
    imports:
      - name: OUTPUT
        package: output-var

Variables that you want to make available to other packages are in the export block of the Zarf package to export a variable from. By default, all exported variables are available to all of the packages in a bundle. To have another package ingest a specific exported variable, like in the case of variable name collisions, use the imports key to name both the variable and package that the variable is exported from, like in the example above.

In the example above, the OUTPUT variable is created as part of a Zarf Action in the output-var package, and the receive-var package expects a variable called OUTPUT.

Sharing Variables Across Multiple Packages

If a Zarf variable has the same name in multiple packages and you don't want to set it multiple times via the import/export syntax, you can set an environment variable prefixed with UDS_ and it will be applied to all the Zarf packages in a bundle. For example, if multiple packages require a DOMAIN variable, you could set it once with a UDS_DOMAIN environment variable and it would be applied to all packages. Note that this can also be done with the shared key in the uds-config.yaml file.

On deploy, you can also set package variables by using the --set flag. If the package name isn't included in the key (example: --set super=true) the variable will get applied to all of the packages. If the package name is included in the key (example: --set cool-package.super=true) the variable will only get applied to that package.

Variable Precedence and Specificity

In a bundle, variables can come from 4 sources. Those sources and their precedence are shown below in order of least to most specificity:

  • Variables declared in a Zarf pkg
  • Variables import'ed from a bundle package's export
  • Variables configured in the shared key in a uds-config.yaml
  • Variables configured in the variables key in a uds-config.yaml
  • Variables set with an environment variable prefixed with UDS_ (ex. UDS_OUTPUT)
  • Variables set using the --set flag when running the uds deploy command

That is to say, variables set using the --set flag take precedence over all other variable sources.

Duplicate Packages And Naming

It is possible to deploy multiple instances of the same Zarf package in a bundle. For example, the following uds-bundle.yaml deploys 3 instances of the helm-overrides Zarf packags:

kind: UDSBundle
metadata:
   name: duplicates
   description: testing a bundle with duplicate packages in specified namespaces
   version: 0.0.1

packages:
   - name: helm-overrides
     repository: localhost:5000/helm-overrides
     ref: 0.0.1
     overrides:
        podinfo-component:
           unicorn-podinfo: # name of Helm chart
              namespace: podinfo-ns

   # note the unique name and namespace
   - name: helm-overrides-duplicate
     repository: localhost:5000/helm-overrides
     ref: 0.0.1
     overrides:
        podinfo-component:
           unicorn-podinfo:
              namespace: another-podinfo-ns

   # note the unique name, namespace and the path to the Zarf package tarball
   - name: helm-overrides-local-duplicate
     path: src/test/packages/helm/zarf-package-helm-overrides-arm64-0.0.1.tar.zst
     ref: 0.0.1
     overrides:
        podinfo-component:
           unicorn-podinfo:
              namespace: yet-another-podinfo-ns

The naming conventions for deploying duplicate packages are as follows:

  1. The name field of the package in the uds-bundle.yaml must be unique
  2. The duplicate packages must be deployed in different namespaces
  3. In order to deploy duplicates of local packages, the path field must point to a Zarf package tarball instead of to a folder.

Note

Today the duplicate packages feature is only supported for packages with Helm charts. This is because Helm charts' namespaces can be overridden at deploy time.

Zarf Integration

UDS CLI includes a vendored version of Zarf inside of its binary. To use Zarf, simply run uds zarf <command>. For example, to create a Zarf package, run uds zarf create <dir>, or to use the airgap tooling that Zarf provides, run uds zarf tools <cmd>.

Dev Mode

Note

Dev mode is a BETA feature and currently only works with local bundles

Dev mode facilitates faster dev cycles when developing and testing bundles

uds dev deploy <path-to-bundle-yaml-dir>

The dev deploy command performs the following operations

  • Creates Zarf packages for all local packages in a bundle
    • Creates the Zarf tarball in the same directory as the zarf.yaml
    • Will only create the Zarf tarball if one does not already exist
    • Ignores any kind: ZarfInitConfig packages in the bundle
  • Creates a bundle from the newly created Zarf packages
  • Deploys the bundle in YOLO mode, eliminating the need to do a zarf init

uds-cli's People

Contributors

renovate[bot] avatar unclegedd avatar decleaver avatar naveensrinivasan avatar corang avatar tristanholaday avatar zachariahmiller avatar anthonywendt avatar mjnagel avatar kirankt avatar zachgallagher avatar racer159 avatar mikevanhemert avatar jeff-mccoy avatar andrewg-xyz avatar austinabro321 avatar justin-o12 avatar rjferguson21 avatar timblaktu avatar noxsios avatar

Stargazers

 avatar Gavin Scallon avatar Devin Allen avatar Palassis avatar  avatar  avatar Allen Conlon avatar Justin Bailey avatar Justin Law avatar Zack A avatar  avatar  avatar Austen Bryan avatar  avatar  avatar  avatar Noah avatar  avatar Lucas Rodriguez avatar  avatar

Watchers

Blake Burkhart avatar Josh Oates avatar Thomas Runyon avatar  avatar  avatar

uds-cli's Issues

Preflight checks for deployments

As a Delivery engineer
I want to ensure that before I deploy a UDS package, there is an automated preflight check
So that I don't inadvertently deploy a UDS package to the wrong env

Notes

Preflight check includes:

  • Check for AWS connectivity
  • Check for cluster connectivity
  • Check for tool versions (if necessary)
  • Notify the user of connectivity status before continuing the deploy

SBOMs aren't accessible to end users

I don't know where the sboms get sent but I don't see them when I run a bundle deploy.

In an ideal world uds-bundle would aggregate all SBOMs into a single html viewer on create and put it in an easily accessible location on deploy.

Unable to deploy a bundle from oci

Unable to run uds bundle deploy oci://ghcr.io/defenseunicorns/uds-package/software-factory-demo:0.0.1-amd64.

We get the following error.
Failed to deploy bundle: unable to load the Zarf Package: unable to read the zarf.yaml in /tmp/zarf-3271840311: open /tmp/zarf-3271840311/zarf.yaml: no such file or directory

I created this branch with this test that demonstrates the failure we are seeing.

Screenshot 2023-09-11 at 9 44 03 PM

bug: `uds bundle create` adding architecture

I want to create bundles for a different architecture than where the command is being execute. (i.e. build and amd64 bundle from an arm64 machine)

With uds-bundle.yaml

kind: UDSBundle
metadata:
  name: example
  description: an example UDS bundle
  version: 0.0.1

zarf-packages:
  - name: init 
    repository: ghcr.io/defenseunicorns/packages/init
    ref: v0.29.2-amd64
    optional-components:
      - git-server

Running uds bundle create
image
ERROR: Failed to create bundle: ghcr.io/defenseunicorns/packages/rook-ceph:0.0.2-amd64-arm64: not found
adding the extra arm64

UDS-cli Version: v0.0.5-alpha
Running on Apple M2

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

github-actions
.github/actions/golang/action.yaml
  • actions/setup-go v4.1.0@93397bea11091df50f3d7e59dc26a7711a8bcfbe
.github/actions/install-tools/action.yaml
  • sigstore/cosign-installer v3.1.2@11086d25041f77fe8fe7b9ea4e48e3b9192b8f19
  • anchore/sbom-action v0.14.3@78fc58e266e87a38d4194b2137a3d4e9bcaf7ca1
  • docker/setup-buildx-action v3.0.0@f95db51fddba0c2d1ec667646a06c2ce06100226
.github/actions/save-logs/action.yaml
  • actions/upload-artifact v3.1.3@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
.github/actions/zarf/action.yaml
.github/workflows/release.yaml
  • actions/checkout v4.1.0@8ade135a41bc03ea155e62e844d188df1ea18608
  • actions/upload-artifact v3.1.3@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
  • actions/checkout v4.1.0@8ade135a41bc03ea155e62e844d188df1ea18608
  • actions/download-artifact v3.0.2@9bc31d5ccc31df68ecc42ccf4149144866c47d8a
  • actions/checkout v4.1.0@8ade135a41bc03ea155e62e844d188df1ea18608
  • actions/download-artifact v3.0.2@9bc31d5ccc31df68ecc42ccf4149144866c47d8a
  • goreleaser/goreleaser-action v5.0.0@7ec5c2b0c6cdda6e8bbb49444bc797dd33d74dd8
.github/workflows/scan-codeql.yaml
  • actions/checkout v4.1.0@8ade135a41bc03ea155e62e844d188df1ea18608
  • github/codeql-action v2.22.3@0116bc2df50751f9724a2e35ef1f24d22f90e4e1
  • github/codeql-action v2.22.3@0116bc2df50751f9724a2e35ef1f24d22f90e4e1
.github/workflows/scan-lint.yaml
  • actions/checkout v4.1.0@8ade135a41bc03ea155e62e844d188df1ea18608
  • morphy/revive-action v2
.github/workflows/scorecard.yaml
  • actions/checkout v4.1.0@8ade135a41bc03ea155e62e844d188df1ea18608
  • ossf/scorecard-action v2.3.0@483ef80eb98fb506c348f7d62e28055e49fe2398
  • actions/upload-artifact v3.1.3@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
  • github/codeql-action v2.22.3@0116bc2df50751f9724a2e35ef1f24d22f90e4e1
.github/workflows/test-e2e.yaml
  • actions/checkout v4.1.0@8ade135a41bc03ea155e62e844d188df1ea18608
.github/workflows/test-schema.yaml
  • actions/checkout v4.1.0@8ade135a41bc03ea155e62e844d188df1ea18608
.github/workflows/test-unit.yaml
  • actions/checkout v4.1.0@8ade135a41bc03ea155e62e844d188df1ea18608
gomod
go.mod
  • go 1.21
  • github.com/AlecAivazis/survey/v2 v2.3.7
  • github.com/alecthomas/jsonschema v0.0.0-20220216202328-9eeeec9d044b@9eeeec9d044b
  • github.com/defenseunicorns/zarf v0.29.1
  • github.com/goccy/go-yaml v1.11.2
  • github.com/mholt/archiver/v3 v3.5.1
  • github.com/mholt/archiver/v4 v4.0.0-alpha.8
  • github.com/opencontainers/image-spec v1.1.0-rc5
  • github.com/pterm/pterm v0.12.69
  • github.com/spf13/cobra v1.7.0
  • github.com/spf13/viper v1.17.0
  • github.com/stretchr/testify v1.8.4
  • golang.org/x/exp v0.0.0-20231006140011-7918f672742d@7918f672742d
  • golang.org/x/sync v0.4.0
  • oras.land/oras-go/v2 v2.2.1

  • Check this box to trigger a request for Renovate to run again on this repository

Bundle local Zarf pkgs

As a UDS Bundle creator
I want to be able to test with local Zarf pkgs on my filesystem
So that I can easily test and debug bundles

Standardize and enforce import syntax

Currently our import syntax is all over the place. Recommended standardizing on goimports-style of imports and enforcing that in CI and/or precommit

Give user idea of progress while pulling oci layers

Just showing layer hashes doesn't give the user any idea of actual progress of said pull. Possible solutions are counting bytes in directory layers are being downloaded to or even just a counter of layers downloaded vs total layers.

More/better progress indicators

There are a few places during bundle create and deploy that uds-cli is doing something but isn't showing any form of working or progress during said operation, this can be concerning to end users, and is especially bad with large bundles/slow hardware.

ADR for 'ref' field in uds-bundle.yaml

We've received a ton of feedback regarding the ref field in uds-bundle.yaml files. The current behavior is that users specify a ref as something like "0.0.1" but the UDS CLI mutates that ref to be "0.0.1-amd64", which can be confusing.

Write an ADR for how we want to handle the ref field

Publish cmd

As a bundle creator
I want to be able to publish bundles that point to both local and remote Zarf packages
So that I can distribute my bundle

Dogfood UDS CLI

Build a small project/repo that deploys a UDS Bundle to multiple envs using CI, etc. Experiment with structure, patterns, etc and form an opinion on best practices / preferred usage

Revert to `bundle` syntax in CLI

This is something I feel rather strongly about and not a case of "less is more". While currently the only functionality of uds-cli is for managing bundles, I think it's rather agreed upon that there will be other functionality integrated into uds-cli as time progresses. This is implied here by @jeff-mccoy.

I am of the opinion that not enough time/input was allowed for the decision of deprecating the bundle syntax given that it seems that most of the feedback from the people using uds-cli was, at least initially, to not do this.

The arguments that I've seen for this seem to just boil down to "less typing is always better" and "less words is better ux". While there are cases where this is good I don't think the difference between uds b c and uds c (you'd be using the shorthand if character count matters to you) is worth it. I also don't think removing the bundle word is better ux. There have been a few examples of other applications given as basis for why uds-cli should remove the bundle noun (helm install , tf apply) but I don't think those examples have the same use case as uds-cli.

This part is hard to articulate but those applications work like that because they are single purpose, helm installs helm charts, terraform applies terraform. While uds-cli is single purpose today, as I referenced above, I'm almost positive it will not be, and even from a naming perspective it doesn't make sense. helm install works because you're installing a "helm", tf apply works because you're applying terraform, uds create is not intuitive and doesn't make sense because you're not "creating a uds" you're creating a bundle. If the sole purpose of this application was for bundles and was named appropriately I wouldn't have any issues with it, but as it stands I think at best this change will be reverted relatively soon once any other functionality is added. And at worst if more functionality is added and this syntax stays it will only serve to confuse end users and make the application less appealing.

Unnecessary printing of "Saving log to file"

when running uds version it doesn't need to save a log and print the log location to screen

corang@halley-server:~/swf/uds-package-software-factory$ uds version

 NOTE  Saving log file to /tmp/zarf-2023-08-29-16-49-26-4239909307.log
v0.0.3-alpha

zarf behaviour

corang@halley-server:~/swf/uds-package-software-factory$ zarf version
v0.29.0

Allow overriding a zarf package's source at deploy time

Context

When developing a bundle using remote packages, the cycle time and process for consuming upstream changes can be cumbersome. If a bundle consumes a tagged and published Zarf artifact, absent other workarounds, the developer must publish a new artifact before being able to test the change to the bundle. That can include significant time sinks (code review, automated tests, PR process etc) in the individual package repo and significantly slow down the dev cycle for a bundle.

Feature Request

Allow the user to override a package repository source at deploy time to leverage a local build path for the given package.

Example

Given:

kind: UDSBundle
metadata:
  name: software-factory-nutanix
  description: A UDS bundle for deploying a software factory to an RKE2 cluster
  version: 0.0.1
  architecture: amd64

zarf-packages:
  # Zarf init
  - name: init
    repository: ghcr.io/defenseunicorns/packages/init
    ref: v0.29.1
    optional-components:
      - git-server

  # Gitlab
  - name: gitlab-redis
    repository: ghcr.io/defenseunicorns/uds-capability/gitlab/dev-dependency/gitlab-redis
    ref: 0.0.2

Allow the user to replace a bundled package at deploy time with something like --local-package-redirect gitlab-redis=<path_to_local_zarf_package_dir>

In this way, a user could develop and test changes to component packages in the bundle and ensure they are functional prior to working through the change process in the other repo.

Test GHCR

As a UDS-CLI dev
I want to test Bundles with Zarf packages that are in GHCR
So I can be confident UDS-CLI is compatible with GHCR

Notes

  • Recommend pulling the Zarf init pkg from GHCR as part of src/test/packags/01-uds-bundle and using that in one of the existing tests

Unable to resume a bundle remove or bundle deploy from where it left off

If a uds bundle deploy or uds bundle resume fails for any number of reasons (for example session timeout on aws token) there is currently not a way to resume. On deploy, it will redeploy all packages even if they have already been deployed with no diff. On remove, the bundle will fail to continue removing as it cannot find the initial components to remove.

Bundles can take a very long time to deploy or remove and as such, not having the ability to resume deploy and remove operations costs a lot of time to fix if they do not run to completion for any reason.

Support setting package variables

Currently I believe there are two ways to set variables for zarf packages included in a bundle:

  1. Via the uds-config.yaml
bundle:
  deploy:
    zarf-packages:
      dubbd:
        set:
          FOO: "bar"
  1. Via imports/exports

I would propose that we support two additional methods of specifying variables

  • Accept --set argument to bundle deploy to such that you can do something like uds bundle deploy --set dubbd.FOO=bar
  • Accept defining variables in the uds-bundle.yaml similar to what is possible with a zarf.yaml:
kind: UDSBundle
metadata:
  name: uds-core
  description: uds core
  version: 0.0.1

zarf-packages:
  - name: dubbd
    repository: oci://ghcr.io/defenseunicorns/packages/dubbd
    variables:
      - name: FOO
        default: bar

Renovate Dashboard ๐Ÿค–

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

These problems occurred while renovating this repository. View logs.

  • WARN: Found renovate config warnings

Edited/Blocked

These updates have been manually edited so Renovate will no longer make changes. To discard all commits and start over, click on a checkbox.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

github-actions
.github/actions/golang/action.yaml
  • actions/setup-go v5.0.1@cdcb36043654635271a94b9a6d1392de5bb323a7
.github/actions/install-tools/action.yaml
  • sigstore/cosign-installer v3.5.0@59acb6260d9c0ba8f4a2f9d9b48431a222b68e20
  • anchore/sbom-action v0.15.11@7ccf588e3cf3cc2611714c2eeae48550fbc17552
  • docker/setup-buildx-action v3.3.0@d70bba72b1f3fd22344832f00baa16ece964efeb
.github/actions/install-uds-cli/action.yaml
  • Homebrew/actions 4b34604e75af8f8b23b454f0b5ffb7c5d8ce0056
.github/actions/save-logs/action.yaml
  • actions/upload-artifact v4.3.3@65462800fd760344b1a7b4382951275a0abb4808
.github/workflows/commitlint.yaml
  • actions/checkout v4.1.5@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b
  • actions/setup-node v4.0.2@60edb5dd545a775178f52524783378180af0d1f8
.github/workflows/nightly-ghcr.yaml
  • actions/checkout v4.1.5@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b
  • docker/login-action v3.1.0@e92390c5fb421da1463c202d546fed0ec5c39f20
.github/workflows/nightly-uds-core.yaml
  • actions/checkout v4.1.5@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b
.github/workflows/release.yaml
  • actions/checkout v4.1.5@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b
  • actions/upload-artifact v4.3.3@65462800fd760344b1a7b4382951275a0abb4808
  • actions/checkout v4.1.5@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b
  • actions/download-artifact v4.1.7@65a9edc5881444af0b9093a5e628f2fe47ea3b2e
  • docker/login-action v3.1.0@e92390c5fb421da1463c202d546fed0ec5c39f20
  • actions/checkout v4.1.5@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b
  • actions/download-artifact v4.1.7@65a9edc5881444af0b9093a5e628f2fe47ea3b2e
  • peter-murray/workflow-application-token-action v3.0.0@dc0413987a085fa17d19df9e47d4677cf81ffef3
  • goreleaser/goreleaser-action v5.1.0@5742e2a039330cbb23ebf35f046f814d4c6ff811
.github/workflows/scan-codeql.yaml
  • actions/checkout v4.1.5@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b
  • github/codeql-action v3.25.5@b7cec7526559c32f1616476ff32d17ba4c59b2d6
  • github/codeql-action v3.25.5@b7cec7526559c32f1616476ff32d17ba4c59b2d6
.github/workflows/scan-lint.yaml
  • actions/checkout v4.1.5@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b
  • pre-commit/action f7acafac0271bdd064cdfa1b13f17b4350e565ed
  • morphy/revive-action v2@sha256:087d4e61077087755711ab7e9fae3cc899b7bb07ff8f6a30c3dfb240b1620ae8
.github/workflows/scorecard.yaml
  • actions/checkout v4.1.5@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b
  • ossf/scorecard-action v2.3.3@dc50aa9510b46c811795eb24b2f1ba02a914e534
  • actions/upload-artifact v4.3.3@65462800fd760344b1a7b4382951275a0abb4808
  • github/codeql-action v3.25.5@b7cec7526559c32f1616476ff32d17ba4c59b2d6
.github/workflows/test-e2e-pr.yaml
  • actions/checkout v4.1.5@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b
.github/workflows/test-schema.yaml
  • actions/checkout v4.1.5@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b
.github/workflows/test-unit.yaml
  • actions/checkout v4.1.5@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b
gomod
go.mod
  • go 1.21.10
  • github.com/AlecAivazis/survey/v2 v2.3.7
  • github.com/alecthomas/jsonschema v0.0.0-20220216202328-9eeeec9d044b@9eeeec9d044b
  • github.com/charmbracelet/bubbles v0.18.0
  • github.com/charmbracelet/bubbletea v0.25.0
  • github.com/charmbracelet/lipgloss v0.10.0
  • github.com/defenseunicorns/maru-runner v0.1.1
  • github.com/defenseunicorns/pkg/helpers v1.1.2
  • github.com/defenseunicorns/pkg/oci v0.0.2
  • github.com/defenseunicorns/zarf v0.33.0
  • github.com/fatih/color v1.16.0
  • github.com/fsnotify/fsnotify v1.7.0
  • github.com/goccy/go-yaml v1.11.3
  • github.com/mholt/archiver/v3 v3.5.1
  • github.com/mholt/archiver/v4 v4.0.0-alpha.8
  • github.com/opencontainers/image-spec v1.1.0
  • github.com/pterm/pterm v0.12.79
  • github.com/spf13/cobra v1.8.0
  • github.com/spf13/viper v1.18.2
  • github.com/stretchr/testify v1.9.0
  • golang.org/x/exp v0.0.0-20240506185415-9bf2ced13842@9bf2ced13842
  • golang.org/x/mod v0.17.0
  • golang.org/x/sync v0.7.0
  • golang.org/x/term v0.20.0
  • helm.sh/helm/v3 v3.14.4
  • oras.land/oras-go/v2 v2.5.0
helmv3
src/test/packages/helm/unicorn-podinfo/Chart.yaml
  • podinfo 6.6.2
regex
src/test/bundles/04-init/uds-bundle.yaml
  • defenseunicorns/zarf v0.33.0
  • defenseunicorns/zarf v0.33.0
src/test/packages/helm/zarf.yaml
  • stefanprodan/podinfo 6.6.2

  • Check this box to trigger a request for Renovate to run again on this repository

Add documentation

As a UDS package creator
I want to know how to use UDS-CLI
To deploy multiple Zarf packages

Allow targeted deploy of specific packages with `--zarf-packages`

Context

When developing (and especially debugging) larger bundles, it can be very time consuming to test a change to a package that falls towards the end of the list. Even if it's a simple config change, we still have to wait for all the other packages to redeploy. It would be helpful to be able to target specific packages to deploy.

Feature Request

Add a --zarf-packages flag which accepts a CSV list of package names to deploy, similar to the functionality of Zarf's --components flag (documented here.

Requirements

  • The specified package names must be defined in the bundle
  • The user is responsible for managing variable dependencies (i.e. if package B requires an exported variable from package A, the user is expected to know and provide both)

Add UDS CLI to brew

Our org maintains a brew tap, add UDS CLI release binaries via the release automation already in place (goreleaser)

Add feature for including skeletons in a bundle

Slack thread with a bunch of context: https://defense-unicorns.slack.com/archives/C029NJU2S4B/p1696012055101239

Basically if uds products want to have reusable capability dependency packages they need to be specified as skeletons and not as built packages. This is technically currently possible through a bunch of extra steps that I think would make more sense to be internalized into uds-cli. Current workaround implementation:

  1. Have a uds-bundle.yaml
  2. have that bundle reference 2 local packages
  3. each local package (zarf.yaml) references the same skeleton
  4. each local package has a different zarf-config.yaml that sets different values for the TMPLs in said skeleton
  5. build each package to the directory the bundle expects
  6. build bundle

I think a limited implementation of skeletons in bundles could work really well with a syntax like this

kind: UDSBundle
metadata:
  name: bundle-of-skeletons

zarf-packages:
  - name: planet-mars
    repository: ghcr.io/defenseunicorns/planet
    ref: 1.0
    skeleton: true
    package-templates:
      - name: PLANET
        value: mars
  - name: planet-neptune
    repository: ghcr.io/defenseunicorns/planet
    ref: 1.0
    skeleton: true
    package-templates:
      - name: PLANET
        value: neptune

This obviously does not afford the full flexibility of skeletons (composing and overrides) but gives a very simple path to creating multi-use/reusable package definitions for common needs. A database being a prime example.

You would just run uds bundle create as usual and it would handle the building of any skeletons in the bundle for you and then just treat them normally.

Allow config file overlays

Context

When managing configurations for multiple environments, there are often many duplicated settings/values across each configuration. A common pattern for simplifying the management of these kinds of configurations (used by Helm, Kustomize and others) is an overlay system, wherein multiple configuration files are provided at once, and values are set in priority order by whichever file "most recently" set them. This allows users to manage a set of base configurations with common values, and separate individual configurations for environment specific, or sensitive values.

Feature Request

  • Allow the user to specify configuration files at deploy time via command line flag
  • Collate the values into a single config

Reuse Init pkg during tests

As a UDS-CLI dev
I want to reuse the init package deployed in the k3d cluster during testing
So that I don't waste time removing/deploying the init pkg over and over, and my e2e tests are more efficient

Notes

  • Recommend testing Remove functionality once, and then the next test that installs the Zarf init package shouldn't remove it
  • After that, I recommend using the src/test/packages/zarf/podinfo Zarf package for testing. Feel free to use some other (small) app as well alongside podinfo. Mentioning this because currently podinfo deploys alongside the init pkg, which will already have been installed

Refine Renovate Configs

A Renovate config has been added to the repo but we need to take some time further refine and think through its behavior (think like automerging patch updates, etc)

Create Bundles Locally

As a UDS Bundle creator
I want to create bundles locally as opposed to in an OCI registry
So that I can have quicker feedback loops during Bundle dev

Use Zarf Variables Per Package

As a UDS Bundle creator
I want to be able to pass variables to the Zarf packages inside the UDS Bundle
So that I can configure each Zarf package for my environment

Revisit Cue

Many of the variables issues that are cropping (global vars in particular) may be solved with Cue

You can't export variables with the same name across multiple packages

Because of how uds-cli stores exported variables having 2 exports with the same variable name doesn't work since the next occurrence of an export overwrites the first one. Simplified version of erroring bundle below

kind: UDSBundle
metadata:
  name: software-factory-demo
  description: A UDS bundle for deploying a software factory to k3d for demonstration purposes NOT FOR PRODUCTION
  version: 0.0.2
  architecture: amd64

zarf-packages:
  - name: software-factory-idam-gitlab
    path: build
    ref: 1.0.0
    exports:
      - name: IDAM_ENABLED

  - name: software-factory-idam-sonarqube
    path: build
    ref: 1.0.0
    exports:
      - name: IDAM_ENABLED

  - name: gitlab
    repository: ghcr.io/defenseunicorns/uds-capability/gitlab
    ref: 0.0.9
    imports:
      - name: IDAM_ENABLED
        package: software-factory-idam-gitlab

  - name: sonarqube
    repository: ghcr.io/defenseunicorns/uds-capability/sonarqube
    ref: 0.0.8
    imports:
      - name: IDAM_ENABLED
        package: software-factory-idam-sonarqube

Error:

Failed to create bundle: error validating bundle vars: import var IDAM_ENABLED does not have a
             matching export

Upgrade to latest Zarf

Starting at v0.30.0, Zarf introduces breaking changes to its library code allowing for more customization around deploys. Investigate and implement changes to Zarf library usage

Order of Removing Deployed Bundle Packages

Problem: When bundling and deploying multiple packages they are deployed in the order they are defined in uds-bundle.yaml. However when removing the bundle using uds bundle remove <bundle>.tar.zst --confirm also removes those packages in the same order, rather than in a first in last out order.

A user should never be able to select which components get deployed from a package in a bundle when `optional-components` is declared

Error log:

                                                                                                      
  ๐Ÿ“ฆ LOGGING COMPONENT                                                                                
                                                                                                      

     ERROR:  Failed to deploy bundle: unable to deploy all components in this Zarf Package: unable to deploy
             component logging: unable to deploy component logging: unable to push images to the registry: open
             /tmp/zarf-2357824383/images/blobs/sha256/0abe676a9b183901b27c89d50397611e820be40583fe02ef94e148fe33956f00:
             no such file or directory

Bundle spec:

zarf-packages:
  # Zarf init
  - name: init
    repository: ghcr.io/defenseunicorns/packages/init
    ref: v0.29.1
    optional-components:
      - git-server

Deploy without confirm and select logging as a component you want.

Bundles are meant to be declaritive, by allowing user choice of components in packages during deploy time you lose that declaritiveness. I think a user should never be allowed to select components but I'll settle for not allowing when optional-components is specified.

Use OCI concurrency and caching where possible

Caching and OCI concurrency have recently been implemented for pull and publish ops, this enhancement also needs to be implemented for bundle creation and any other places where we are pulling from OCI

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.