defenseunicorns / zarf Goto Github PK

The Tiny-Kafka example doesn't work. zarf package create works fine, but when running zarf package deploy I get the error message:

INFO[0008] Copying file                                  Destination=/var/lib/rancher/k3s/server/static/charts/strimzi-kafka-operator-0.24.0.tgz Source=/tmp/zarf-019814346/charts/strimzi-kafka-operator-0.24.0.tgz
FATA[0008] Unable to copy the contents of the asset      Destination=/var/lib/rancher/k3s/server/static/charts/strimzi-kafka-operator-0.24.0.tgz Source=/tmp/zarf-019814346/charts/strimzi-kafka-operator-0.24.0.tgz

Digging deeper, it looks to be because the .tgz file that Strimzi publishes is actually named strimzi-kafka-operator-helm-3-chart-0.24.0.tgz, and that's what gets saved in the tarball, but Zarf expects the chart files to be named <ChartName>-<Version>.tgz

I'm submitting a PR to fix the example by pulling it down under files: instead of charts: but it's just a bandaid fix and deserves to be looked at more closely.

Most IB images currently are either missing or the version is missing that we need. Additionally, some nearly all of the IB images are baselined off of UBI when they could easily run on a distroless IB baseline instead and cut the overall size significantly.

Migrated from https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/-/issues/16

It's probably also worth discussing the use if interal K8s routes for git and zarf.localhost for the registry mirror. Ideally they'd both use HTTPS and the ingress consistently.

Originally posted by @jeff-mccoy in #29 (comment)

We need to change the paths for git repositories from an East-West URL (*.<namespace>.svc.cluster.local) to one that comes in on the Ingress for Gitea (https://zarf.localhost)

Watch out for possible certificate/CA trust issues

Zarf packages should be able to be signed by a package creator. Signing keys could come from GPG, a smart card, X509 key, etc. Signatures can be added in a block to the zarf.yaml. A zarf package config with signatures could look something like:

kind: ZarfPackageConfig
metadata:
  name: appliance-demo-doom
  description: "Demo Zarf appliance mode with doom game"

local:
  manifests: manifests

  images:
    - registry.dso.mil/platform-one/big-bang/apps/product-tools/zarf/game:doom
signatures:
  - keyID: <KEY ID>
    signature: <SIGNATURE>

<KEY ID> is a hash of the signing key's public key and the signature is, obviously, the signature.
Fairly straight forward. The signature could be of the zarf yaml outside of the signatures block. If this creates difficulty signatures could exist in a separate file.

These signatures then should be used to verify the package during the package deploy stage.

When signed with a key backed by PKI such as a CAC the signature should also be verified against that PKI's CA.

Implement hashing/signing for all elements of the package to ensure authenticity and integrity of the artifacts.

Migrated from https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/-/issues/36

Helm 3.7 release is now out with breaking changes for packages. We should evaluate this as we were waiting for this to deal with a prior issue with the oras dep in 3.6 (which is why we are currently on v3.6.1-0.20210719095234-663c5698878c).

The remote link for the kubescape binary seems to be mutable therefore invalidating the SHA256 hash.

Removing all hard-coded elements of K3s will enable transparent support for alternative cluster technologies including K3d to make development and testing simpler. This will likely require #23 to be implemented.

Migrated from https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/-/issues/31

As a developer of Zarf, I want the E2E tests to happen on an ephemeral server, so that I can ensure that they are always running on a clean environment and so more than one test suite can run concurrently

Migrated from https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/-/issues/34

Currently hashes are being collected for some resources but not all. This should be expanded to include all images, helm charts, files, etc. When resources are fetched from a remote hashes should be confirmed with the remote registry after pull. These hashes should then be verified prior to deploying a zarf package.

This likely will be a sub-task of #22

The game example has an image it needs to run the demo that we need to move off of Repo1 container registry to turn off that feature in the project mirror.

https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/container_registry

At the very least, the utility registry should require credentials to push images. Ideally these creds would be dynamically handled similarly to #3.

Migrated from https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/-/issues/9

Currently the shasum registered in the config.yaml does nothing, this needs to actually perform a validation after file download. This should also be used for deployment.

• Compute downloaded file shasum
• Verify shasum in config.yaml against computed valued
• During deployment, compute shasum against the extracted file in the archive prior to placement

Migrated from https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/-/issues/17

Integrating pre-commit hooks would let us validate that linting has been done, along with a host of other good checks, like automated protection from accidentally committing some secrets, automated style guide, automated security scanning of golang and terraform code, etc

Need to cleanup the README and asciinema to use the latest version of zarf.

If I put a "short" version of a docker image in the remote: section, it throws an error when running zarf package create zarf package deploy:

Changing it to fully qualified e.g. docker.io/rancher/coredns-coredns:1.8.3 works fine.

Evaluate tools / options to produce a standard consumable SBOM and a user-friendly display / ability to navigate it when deploying a zarf package. Doing so will meet software supply chain requirements for government and industry as well as provide a higher level of transparency/confidence to what is transported with Zarf.

Related:
https://www.ntia.doc.gov/files/ntia/publications/sbom_minimum_elements_report.pdf
https://cyclonedx.org/
https://spdx.dev/
https://github.com/anchore/syft

Migrated from https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/-/issues/35

For greater flexibility, change from K3s manifest applier to zarf applying manifests. Because CRDs will also require some concept of ordering, we should evaluate how Helm does this and use that if possible. The client-go APIs are very low-level for the complexity of this operation.

Migrated from https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/-/issues/30

As a user of Zarf I'd like zarf prepare sha256sum to work with remote objects, so that I don't have to manually download the file I want to checksum before adding it to my zarf.yaml

Example:

root@zarf-examples:/home/vagrant/zarf-examples# ./zarf prepare sha256sum https://archive.apache.org/dist/kafka/2.8.0/kafka_2.13-2.8.0.tgz

         *,
         *(((&&&&&/*.
          *(((((%&&&&&&&*,
           *(((((((&&&&&&&&&*              ,,*****,.                      **%&&&&&((((((
            *(((((((((&&&&&&&@*    **@@@@@@&&&&&&&&&&@@@@@**         */&&&&&&((((((((((
              *((((((///(&&&&&&@@@@&&&&@@@@@@@@@&&&&&&&&&&&&&&@/* *%&&&&&&/////((((((*
                *(((///////&&&&&&&&&&&&&@@@@@@@@@&&&&&&&&&&&&&&&&&(%&&&/**///////(/*
       */&&&&&&&&&&&&&&&&*/***&%&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&/*******///*
   *&%&&&&&&&&&&&&&&&&&&&&&&&***&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&****/&&&&&&&&&&&&(/*
 */((((((((((((((///////******%%&&&&&&&&//&@@*&&&&&&&&&&&&&&&&&&&#%&&&&*/####(/////((((/.
     */((((((((((///////******%%%%%%%%%(##@@@//%&%%%%%%%&%&%%&&/(@@(/&&(***///////(((*
          ***(((((/////********%%%%%%%/&%***/((%%%%%%%%%%%%%%%%(#&@*%/%%***/////**
            *&&%%%%%%//*******%%%%%%%%@@****%%/%%%%%%%%%%%%%%%%%%***@@%%**(%%%&&*
          *&&%%%%%(////******/(%%%%%%%@@@**@@&%%%%%%%%%%%%%%%%%(@@*%@@%%*****////%&*
        *&%%%%%#////////***/////%%%%%%*@@@@@/%%%%%%%%%%%%%%%%%%%%@@@@%%*****///////((*
       *%%%%((((///////*    *////(%%%%%%##%%%%%%%%%%%(%%%%*%%%%%%%%%%%*
      *(((((((/***            */////#%%%%%%%%%%#%%%%%%%%%%%%%%%%%%%%#*
                   %%(           ,*///((%%%%%%%%(**/#%%%##**/%%%%%*
                 %%%&&&&           *///*/(((((########//######**
                 %&&&&&*          *#######(((((((//////((((*
                                  ###%##############(((#####*
                   %@&&          *&#(%######*#########(#####/
                   /&&* ..       ,&#(/%####(*#########/#####/             #%@%&&&
             **         &&     ./%##((*&####/(#######(#####*(*            %&&&&&&
           *@%%@*             *&#####((((####*(#####(*###(*(##*              ,  %@&
          *@%%%%*            *%######((((*%####/*((*%####/*(###*  *
         *@%%%%%%*      *##* **#(###((((///#*#*(((((/#**#((*(##**#,*/##*,    %@&&
         *@%%%*%%%*  ****,*##/*#*##(((((((/(((((((((/(((*(((((###########*,  #&&#
         *@%%%*(%%%/*   **######(#((..((((((((((((((((((*  ,*(#####(((((*,
         *@%%%#(*%%%%*   ,**/####(* */(((((((((((((((((*     ,**,
          *@%%%*(/(%%%%/*     ******(((((((((((*(((((*
           *@%%%#(((*/%%%%%%##%%*((((((((((((**((((*
            *@%%%%*(((((((((((((((((((((((*/%*((*.             (&&&(
             ,*%%%%%%*((((((((((((((((**%%%**,                (&
                *%%%%%%%%%(/*****(#%%%%%**                      &%
                   ,**%%%%%%%%%%%%%***

             ,((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((,
                         .....(((((((/////////////////((((((((.....

            ///////////////      ///////      *****************  ***************,
                    ////.       ///  ////     *///          ***  ****
                 ////,         ///    ////    *///////////////.  /////**/******
              /////          //////////////   *///      *///     ///*
           ./////////////// ////         ///  *///        ////   ///*



WARNING: This is a remote source. If a published checksum is available you should use that rather than calculating it.
3fa380ae5d1385111ee9c83b0d1806172924ffec2e29399fd1a42671a97492c6
root@zarf-examples:/home/vagrant/zarf-examples#

Migrated from https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/-/issues/13

As a user of Zarf I want a roadmap board available so that I can easily determine what new features may be coming next.

Ref example: https://github.com/aws/containers-roadmap/projects/1

Discussed in #63

features:
  - name: management
    description: "Add the K9s terminal-based K8s UI for cluster management"
    default: true
    files:
      - source: https://zarf-public.s3-us-gov-west-1.amazonaws.com/k9s_Linux_x86_64_v0_24_11
        shasum: 18a5a33bbf58cb228e56a03380dcb6b9bb8624acab4ff63deb7364dc15d3c03f
        target: "/usr/local/bin/k9s"
        executable: true
      - source: assets/misc/k9s-theme.yaml
        target: "/root/.k9s/skin.yml"

kind: ZarfPackageConfig
metadata:
  name: kafka-strimzi-demo
  description: "Demo tiny Zarf Kafka deployment"
  
local:
  manifests: manifests

  charts:
    - name: strimzi-kafka-operator
      url: https://strimzi.io/charts/
      version: 0.24.0

  images:
    - registry1.dso.mil/ironbank/opensource/strimzi/operator:0.24.0
    - registry1.dso.mil/ironbank/opensource/strimzi/kafka:0.24.0-kafka-2.8.0

features:
  - name: kafka-tools
    description: "Add useful tools to manage Kafka"
    default: true
    files:
      # Helper tools for working with kafka
      - source: https://archive.apache.org/dist/kafka/2.8.0/kafka_2.13-2.8.0.tgz
        shasum: 3fa380ae5d1385111ee9c83b0d1806172924ffec2e29399fd1a42671a97492c6
        target: /opt/kafka.tgz
```</div>

Brought to you by 🦄 Dash Days, where we spend 2 days of every month to have fun and get gooder at things™.

As a user of Zarf who needs to run stable, maintainable, performant Postgres database(s) in my edge cluster, I want a new example to look at as a starting-off point so that I can iterate on what has already been looked at rather than starting from scratch

We only use containerd for parsing the docker URI and bring in the containerd library for that. Considering we also go-containertools and other tools in the ecosystem, we should evaluate alternatives for https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/-/blob/master/cli/internal/images/push.go#L6.

Migrated from https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/-/issues/32

Still investigating, but want to get written down what I already have.

Helped a Zarf user just now that had his hostname set to localhost.localdomain and created a package with remote images to be loaded into the utility cluster's image registry, and Zarf was trying to hit the registry at zarf.localhost which didn't work. Here's the rough series of events:

1. He started out with specifying the host as an IP address in zarf init, and was hitting this issue of Zarf being unable to reach the utility cluster's registry
2. We destroyed the cluster and tried again with zarf init --host=localhost, still didn't work
3. We changed his /etc/hosts by adding zarf.localhost to his 127.0.0.1 line - worked great

Still investigating a minimally reproducible example using vagrant. Stay tuned.

"--ignore-missing" for the shasum command is not always available. Need to update docs to address this or possibly embed into the CLI directly

Migrated from https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/-/issues/14

Invalid entries can cause the TLS cert to be improperly formatted. In a test deployment, passing a non-valid hostname from RHEL 7 (in this case containing an underscore ""), broke the PKI trust for Zarf. There is some debate, but the general consensus seems to be don't allow "" to conform to the RFC: https://stackoverflow.com/a/11284416. By just using the function below when a user specifies a hostname on zarf init or zarf pki regenerate.

A host validation function already exists in https://github.com/defenseunicorns/zarf/blob/master/cli/internal/utils/preflight.go#L11 that we should be able to call to test this.

• Validate zarf init hostname when using the prompt
• Validate zarf init --host
• Validate zarf pki regenerate --host

Migrated from https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/-/issues/37

Because Zarf is still making breaking changes in the 0.x release cycle, we should ensure that the given zarf.yaml is compatible with the running zarf binary. Users should be informed if there is a version mismatch that this could cause problems and possible offer a link to the correct version of the binary if not in the airgap. Related, we may need to think about how we bundles updates to the zarf binary itself (can be embedded in the zarf package itself)

As an air gap engineer, I would like to personally review the code changes if there is a merge conflict upon a Big Bang upgrade.

When upgrading Big Bang versions we would like to see a merge request created IF and WHEN a merge conflict is detected. This will allow the user to manually review the conflict and merge in the updates themselves.

Possible solution: when performing the repo.Push() within git.Push, handle the failed push if it is a merge conflict by warning the user, pushing to a new branch in gitea and linking to that branch on the CLI.

MIgrated from https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/-/issues/6

As a user of Zarf who is not in an air gapped situation I would like the ability to deploy a Zarf package that resides in a remote location, so that I don't have to download the package as a separate process

Example:

zarf package deploy https://example.com/path/to/some/package.tar.zst

Brought to you by 🦄 Dash Days, where we spend 2 days of every month to have fun and get gooder at things™.

As a user of Zarf who wants to deploy a stable, performant, highly available, production-like postgres database to an edge cluster, I want the postgres-operator example to include automated backups, so that I have a reference to look at when creating my own package

Our CI pipeline should check for linting errors

Add an update config to the examples folder to demonstrate deploying Big Bang in standard mode to a utility cluster.

Migrated from https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/-/issues/15

Will need to break this out into an epic

Although the Iron Bank lacks significant ARM support at the moment, we only need two images for shift pack's cluster to function (registry and gitea) that could be targeted for Iron Bank ARM support as a priority.

Reference on where to begin:
https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/shift/pack/-/blob/master/Earthfile#L112

Migrated from https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/-/issues/2

Refs:

https://repo1.dso.mil/platform-one/big-bang/bigbang/-/tree/master/docs/airgap
https://confluence.il2.dso.mil/pages/viewpage.action?spaceKey=ONBOARD&title=Edge+Deployment+Scenarios+Defined

Migrated from https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/-/issues/18

As a user of the Shift CLI, when I run a command I would like to get feedback on the command I am running. This will ensure I know the tool is working and not sitting there timing out

I.e when uploading images or repositories

Migrated from https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/-/issues/4

Be cool like @runyontr and use a schema to prevent simple errors with zarf-config.yaml files.

Migrated from https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/-/issues/29

I'm looking at writing a unit test for helping with #45, but because of the use of logContext.Fatal() in the codebase a unit test passes when it shouldn't.

I propose replacing all logContext.Fatal() commands with logContext.Error() and return the error type wherever logContext.Fatal() was being used.

A simple example of this would be to change this:

func Foo() {
  err := doSomethingThatDoesntWork()
  if err != nil {
    logContext.Fatal("some error")
  }
}

to this:

func Foo() error {
  err := doSomethingThatDoesntWork()
  if err != nil {
    e := "some error"
    logContext.Error(e)
    return fmt.Errorf(e)
  }
  return nil
}

When a zarf package is created we should record some attestations about the package's creation. Some attestations could include

• Environment details such as set variables, user, cloud metadata if applicable
• Time of package creation, provided by a trusted timestamp authority
• Any vulnerability or antivirus scans run on the package's contents

We could use some of the features of in-toto as a library. ITE-6 would allow us to create a specific attestation type to zarf packages if required.

Attestations should be displayed to a user deploying the package in a user friendly manner.

What I expected:

As a user of Zarf I want it to give me a simple and descriptive error message when it isn't able to pull container images when creating a package, so that I don't have to track down what my issue is.

What I saw:

In this case it was because registry1.dso.mil was down for a few minutes, but I've also seen this error message when I don't have my credentials for the container registry set properly

 - https://repo1.dso.mil/platform-one/big-bang/apps/core/[email protected]
 - https://repo1.dso.mil/platform-one/big-bang/apps/core/[email protected]
 - https://repo1.dso.mil/platform-one/big-bang/apps/core/[email protected]
 - https://repo1.dso.mil/platform-one/big-bang/apps/security-tools/[email protected]


INFO[0000] Create Zarf package confirmed
INFO[0000] Loading images for local install
INFO[0000] Loading images
INFO[0000] Fetching image metadata                       image="registry1.dso.mil/ironbank/fluxcd/helm-controller:v0.11.0"
WARN[0030] Unable to pull the image                      image="registry1.dso.mil/ironbank/fluxcd/helm-controller:v0.11.0"
INFO[0030] Fetching image metadata                       image="registry1.dso.mil/ironbank/fluxcd/kustomize-controller:v0.13.0"
WARN[0060] Unable to pull the image                      image="registry1.dso.mil/ironbank/fluxcd/kustomize-controller:v0.13.0"
INFO[0060] Fetching image metadata                       image="registry1.dso.mil/ironbank/fluxcd/notification-controller:v0.15.0"
WARN[0091] Unable to pull the image                      image="registry1.dso.mil/ironbank/fluxcd/notification-controller:v0.15.0"
INFO[0091] Fetching image metadata                       image="registry1.dso.mil/ironbank/fluxcd/source-controller:v0.14.0"
WARN[0121] Unable to pull the image                      image="registry1.dso.mil/ironbank/fluxcd/source-controller:v0.14.0"
INFO[0121] Creating image tarball (this will take a while)
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x5a43bf2]

goroutine 1 [running]:
github.com/google/go-containerregistry/pkg/v1/cache.(*image).ConfigName(0xc0004383c0, 0xc0007335c0, 0xc000d1e9e8, 0xc000843d40, 0xc000d1e8a8, 0x42465ff, 0x0)
	<autogenerated>:1 +0x32
github.com/google/go-containerregistry/pkg/v1/tarball.calculateManifest(0xc000d1ecc8, 0x0, 0x0, 0x0, 0x8, 0xc0001140b0)
	/Users/andrewroth/.asdf/installs/golang/1.16.7/packages/pkg/mod/github.com/google/[email protected]/pkg/v1/tarball/write.go:230 +0x143
github.com/google/go-containerregistry/pkg/v1/tarball.getSizeAndManifest(0xc000d1ecc8, 0xc0001140b0, 0xc0001140a8, 0x0, 0x0, 0x70, 0xc000d1ebd0, 0x0, 0x0, 0xc000d1ebe8)
	/Users/andrewroth/.asdf/installs/golang/1.16.7/packages/pkg/mod/github.com/google/[email protected]/pkg/v1/tarball/write.go:298 +0x32
github.com/google/go-containerregistry/pkg/v1/tarball.MultiRefWrite(0xc000d1ecc8, 0x626f940, 0xc0001140a8, 0x0, 0x0, 0x0, 0x0, 0xc000d1edeb)
	/Users/andrewroth/.asdf/installs/golang/1.16.7/packages/pkg/mod/github.com/google/[email protected]/pkg/v1/tarball/write.go:101 +0xc5
github.com/google/go-containerregistry/pkg/v1/tarball.MultiRefWriteToFile(0xc00043c500, 0x50, 0xc000d1ecc8, 0x0, 0x0, 0x0, 0x0, 0x0)
	/Users/andrewroth/.asdf/installs/golang/1.16.7/packages/pkg/mod/github.com/google/[email protected]/pkg/v1/tarball/write.go:64 +0xf2
github.com/google/go-containerregistry/pkg/v1/tarball.MultiWriteToFile(0xc00043c500, 0x50, 0xc000d1efd8, 0x0, 0x0, 0x0, 0xc000754690, 0x0)
	/Users/andrewroth/.asdf/installs/golang/1.16.7/packages/pkg/mod/github.com/google/[email protected]/pkg/v1/tarball/write.go:52 +0x2cb
github.com/google/go-containerregistry/pkg/crane.MultiSave(0xc000d1f5a0, 0xc00043c500, 0x50, 0x1, 0x1)
	/Users/andrewroth/.asdf/installs/golang/1.16.7/packages/pkg/mod/github.com/google/[email protected]/pkg/crane/pull.go:77 +0x66b
repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/cli/internal/images.PullAll(0xc00059d840, 0x4, 0x4, 0xc00043c500, 0x50)
	/Users/andrewroth/src/repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/cli/internal/images/pull.go:29 +0x435
repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/cli/internal/packager.addLocalAssets(0xc0006a76c0, 0x3f, 0xc00043c280, 0x44, 0xc00043c320, 0x45, 0xc00043c410, 0x46, 0xc00043c500, 0x50, ...)
	/Users/andrewroth/src/repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/cli/internal/packager/create.go:125 +0x42a
repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/cli/internal/packager.Create(0x1)
	/Users/andrewroth/src/repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/cli/internal/packager/create.go:37 +0x3fd
repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/cli/cmd.glob..func3(0x726ca20, 0xc000699dd0, 0x0, 0x1)
	/Users/andrewroth/src/repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/cli/cmd/package.go:24 +0x2c
github.com/spf13/cobra.(*Command).execute(0x726ca20, 0xc000699db0, 0x1, 0x1, 0x726ca20, 0xc000699db0)
	/Users/andrewroth/.asdf/installs/golang/1.16.7/packages/pkg/mod/github.com/spf13/[email protected]/command.go:860 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0x726b620, 0xc000010020, 0xc000bbff58, 0x1)
	/Users/andrewroth/.asdf/installs/golang/1.16.7/packages/pkg/mod/github.com/spf13/[email protected]/command.go:974 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
	/Users/andrewroth/.asdf/installs/golang/1.16.7/packages/pkg/mod/github.com/spf13/[email protected]/command.go:902
repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/cli/cmd.Execute()
	/Users/andrewroth/src/repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/cli/cmd/root.go:34 +0xa5
main.main()
	/Users/andrewroth/src/repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/cli/main.go:8 +0x25

Migrated from https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/-/issues/39

As a developer of Zarf I want the ability to build and develop Zarf without having to install the dependencies on my workstation, so that I can manage versions and other dependencies separately from my other projects.

We already use Vagrant for doing demos/examples, let's also set up a Vagrant-based developer environment for developing and building Zarf

This also makes it a lot easier for new developers to get up and running with developing Zarf, by just having them spin up a Vagrant box rather than telling them to go install all these different tools and hope they have the right versions of things.

Co-authored with @btlghrants

As a maintainer of Zarf I want visitors to have Pull Request Templates available so that it is easier to write a high quality pull request

Tiny-kafka example throws error on zarf package create

FATA[0001] Bad HTTP status: 404 Not Found                destination=/var/folders/_d/70qj4sy15hv6tmp2vfpx1khh0000gn/T/zarf-316026052/files/0 url="https://ftp.wayne.edu/apache/kafka/2.8.0/kafka_2.13-2.8.0.tgz

It looks like it is an issue with the upstream at ftp.wayne.edu. Worth evaluating pointing at a different mirror if this one turns out to be flaky?

In the upstream, LFS and git bundle don't work together, but there are some solutions out there that could be a good fit for this. The idea would be to absorb this into Zarf and go-get so it's transparent to consumers pushing data artifacts. Other areas of research here include https://oras.land/, but OCI has proven to be a poor performer in some larger datasets in our early tests.

Issue for Git LFS / Bundle: git-lfs/git-lfs#1755

Migrated from https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/-/issues/20

Okay so I have a really bad idea.... this will be a follow-on to #64, would like to get it complete before we cut this release with breaking changes. This would also help #16 and #17.

Now that we've streamlined much of this config and since @RothAndrew added the containerd mirroring a few weeks ago, I think we can actually really make this better. Basically I'm proposing we:

• Drop the use of containerd imports entirely, they aren't fun and are needlessly different than how we push to the utility cluster registry. This means there would only be one time of "image" in the zarf yaml and it would go to the registry
• Move the docker registry out of the "utility cluster" and make it a required agent for every deployment of zarf (appliance mode or otherwise). The overhead is minuscule and it simplifies a lot of what we do. The Big Bang and Utility Cluster examples use this concept already.
• Drop the "utilityCluster" key completely in the zarf config and allow those to be components as well since there will only be one type of image and repos can just be part of the component definitions
• Change the wording of "utility cluster" to something else like "gitops" or similar since it will now only have gitea, this would really only show up in our docs + the zarf init wizard
• Appliance mode zarf packages wouldn't see any changes to their actual yaml files

What that could look like for the Big Bang example:

As a user of Zarf, I want the ability to provide Kustomizations to be applied, so that I don't have to build the manifests myself using kustomize build

As a maintainer of Zarf I want visitors to have Issue Templates available so it is easy to provide a well formatted bug report or feature request

As a user of Zarf Appliance Mode, I'd like the ability to disable Traefik so that it doesn't conflict with Istio if I'm deploying Big Bang using Zarf.

We should scrub the commit history for old commits and verify their authenticity cryptographically to ensure a clean/validated git history. One possible way to do it: https://superuser.com/a/1123928

Though the underlying components of zarf packaging won't change with diffing, zarf should have a command that a user or pipeline can run that either prompts (or accepts as input) a last commit/tag/amount of time to diff changes to determine what to include in the package. zarf package update should only include those images/repos/files that have changed references since the last defined interval. For many types of data in zarf this is natural as we can treat some things as immutable such as repo and image tags. Although some systems (Iron Bank) do not abide by this convention, it's still the most reliable way to track using just a zarf.yaml via git diffs without more complex logic. Some components though, such as manifests probably are best always packaging as they are very lightweight and declarative.

Migrated from https://repo1.dso.mil/platform-one/big-bang/apps/product-tools/zarf/-/issues/33