slok / kahoy Goto Github PK

At this moment we are not capturing the command execution and is going directly to the stdout/stderr. We should capture this and write inside our logger.

These options are handy to have them in the configuration file, because is likely that don't change in a repository, omitting them from the cmd run, can make it easier to run Kahoy.

👋 With the acquisition of kahoy.dev it would be great to give the project a nice looking docs site.

As we all know writing docs efficiently is extremely complex for a single person or even few people, specially if they hold all the context and lack external feedback.

This issue is to track the progress around the future documentation structure and discussion for the site.

At this moment groups are identified by the base name of a directory (e.g /group-a/subgroup-b/subsubgroup-c would be subsubgroup-c), this makes collisions:

• prometheus/crd
• bilrost/crd

would have collisions.

We should identify these group uniquely based on the path, the group should be identified from the path of the given manifests path, e.g:

• manifests/prometheus/crd: prometheus/crd
• manifests/bilrost/crd: bilrost/crd
• manifests/bilrost/ns: bilrost/ns
• manifests/monitoring/grafana/grafana-dashboards: monitoring/grafana/grafana-dashboards

After #43, the git aware repository should have the context about anything related to git, the fs repository shouldn't, mainly because is the BYO (Bring your own) usage.

Users that want to use git diffs with fs free storage, can continue using it with fs-include filter. To use a full managed git way they should use git storage.

On multi-resource YAML files that start with comments or are empty, YAML parsing fails. Example:

# some comment
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: efs-provisioner
---
...

Parsing this file fails (probably because of # some comment).

Using regexes we can ignore resources based on Kubernetes types (e.g apps/v1/Deployment).

Implementing this we solve this #27

• Features.
• Philosophy.
• Configuration.
• Commands and options.
• Gitops.
• Examples.
• Screenshots and/or gif.
• Logo.
• Docker image ready.
• Alternatives.

Two dry-runs on the same path:

1st execution:

⯈ Apply (10 resources)
├── ⯈ group-a/group-c/group-d (3 resources)
│   ├── apps/v1/Deployment/test-kahoy/app-d1 (group-a/group-c/group-d/d1.yaml)
│   ├── apps/v1/Deployment/test-kahoy/app-d2 (group-a/group-c/group-d/d1.yaml)
│   └── apps/v1/Deployment/test-kahoy/app-d3 (group-a/group-c/group-d/d1.yaml)
├── ⯈ group-a/group-c (1 resources)
│   └── apps/v1/Deployment/test-kahoy/app-c1 (group-a/group-c/c1.yaml)
├── ⯈ group-a (2 resources)
│   ├── apps/v1/Deployment/test-kahoy/app-a1 (group-a/a1.yaml)
│   └── apps/v1/Deployment/test-kahoy/app-a2 (group-a/a1.yaml)
├── ⯈ group-b/group-e (2 resources)
│   ├── apps/v1/Deployment/test-kahoy/app-e1 (group-b/group-e/e1.yaml)
│   └── apps/v1/Deployment/test-kahoy/app-e2 (group-b/group-e/e2.yaml)
├── ⯈ ns (1 resources)
│   └── core/v1/Namespace/default/test-kahoy (ns/ns.yaml)
└── ⯈ root (1 resources)
    └── apps/v1/Deployment/test-kahoy/app-root1 (root1.yaml)

2nd execution:

⯈ Apply (10 resources)
├── ⯈ group-b/group-e (2 resources)
│   ├── apps/v1/Deployment/test-kahoy/app-e1 (group-b/group-e/e1.yaml)
│   └── apps/v1/Deployment/test-kahoy/app-e2 (group-b/group-e/e2.yaml)
├── ⯈ ns (1 resources)
│   └── core/v1/Namespace/default/test-kahoy (ns/ns.yaml)
├── ⯈ root (1 resources)
│   └── apps/v1/Deployment/test-kahoy/app-root1 (root1.yaml)
├── ⯈ group-a/group-c/group-d (3 resources)
│   ├── apps/v1/Deployment/test-kahoy/app-d1 (group-a/group-c/group-d/d1.yaml)
│   ├── apps/v1/Deployment/test-kahoy/app-d2 (group-a/group-c/group-d/d1.yaml)
│   └── apps/v1/Deployment/test-kahoy/app-d3 (group-a/group-c/group-d/d1.yaml)
├── ⯈ group-a/group-c (1 resources)
│   └── apps/v1/Deployment/test-kahoy/app-c1 (group-a/group-c/c1.yaml)
└── ⯈ group-a (2 resources)
    ├── apps/v1/Deployment/test-kahoy/app-a1 (group-a/a1.yaml)
    └── apps/v1/Deployment/test-kahoy/app-a2 (group-a/a1.yaml)

At this moment we are creating the repositories on the main cmd, as dependencies for the planner.

We should have a component that knows how to create both repositories (old and new) and understands how to initialize them.

In the case of fs repositories doesn't have much value, but we could have a factory that understands Git prepares the fs from a git repo, and then can get the required data to create the correct fs repositories.

Mode semantically speaking, is more related to how the execution of kahoy is made (dry-run, diff, regular...). additionally to this, the current mode flag is used to select the provider, that's not intuitive, however, if we rename --mode to --provider would be clear enough.

We should make this before releasing v2 because is a breaking change.

Contributes to #59

When we started introducing priorities, the resources now need to be applied with different politics, we need to implement a way of applying resources in batches.

At this moment the delete of kubectl.DiffManager mimics the diff in client mode (makes the diff locally), this makes that if the resource is not on the cluster, is showing us false diffs.

We could make the delete diff smarter by contacting the server before the diff, and check if the resources are present, in case the resources exist diff them, if not, don't add in the delete diff.

If we use something like -n ./, all resources are detected as if they were in root directory, independently of the subpaths they live on.

The resources should be loaded in their correct group.

When applying resources to a cluster, some of them have requirements over the other e.g: namespaces or CRDs.

We could implement our internal way of ordering the resources to be applied but this is not flexible, so we should have priorities that can be set by the users.

• Apply resources grouped by priorities (#60).
• Add priorities at group level(#58).

We should have a flag in case we want forcing it

At this moment when using git, we are applying only the resources that have changed based on the file system files (git diff).

We could compare them at the resources level and create a more generic one that can be used with any of the repository implementation.

Optionally, Kahoy should have the ability to write a report as an output with the resources Applied and deleted supporting different formats. For example:

• json: A common API in JSON format API.
• kubedog: Format to support multitracker Kubedog format.

With Kubectl implementation we can use --ignore-not-found=true.

Wait is not responsability of Kahoy except on priorities that is a requirement, that's why, delete operation should not wait.

This is related to #105 that will output report data so another tool (e.g Kubectl) can wait for it.

Contributes to #59

Groups could have specific options that are applied to all the resources in the group. e.g:

• Wait for group resources.
• Priorities
  ...

The processors can process resources and use to check the resources themselves or mutate them.

Context

At this moment we are loading the manifests from 2 different paths, planning the states...

We implement this with an fs.Repository type storage.Repository having an old and a new storage.Repository, with both we can get the plan of whats should exist and what should not exist.

On top of these, we have created a factory that knows how to return the old and new repos, this gives us the ability to be smarter when creating both storages.

Problem

The common case for Kahoy is to be used in gitops style, so normally we will have the state on a Git repository, with the paths mode implementation, we can prepare the repositories previously (e.g). But this is messy, having bash scripts around to prepare the

Solution

We could implement a git mode that would make all these preparation inside Kahoy using the repository loaded in memory. We would load 2 times the fs repo, set the old in the before-commit state, and use these two repositories as a regular fs.Repository implementation but from memory, each one in its own git state (emulating two paths).

Basically what the example script does but from memory and inside Kahoy.

Include and exclude options should work at the same time, exclude should have priority.

At this moment we are only unmarshalling regular objects, our code will panic if tries decoding a metav1.List object. We should be able to handle this correctly. These are the object YAML style:

apiVersion: v1                                                                                            
kind: List
items: []

Normally we are setting the same path for old and new in the git mode, if new is passed, this should warn and set the old to the same one.

When executing apply without dry-run or diff, ask if we want to continue. We should add a flag to force so is not interactive.

Tasks:

• When not dry-run or diff modes ask for confirmation to continue the execution.
• Add --yes parameter so that the confirmation can be skipped i.e for CI usecases.

• What is Kahoy
• What can you do with Kahoy
• Kahoy Requirements and Limitations
• Alternatives

In the cases where the namespace does not exists and we do a regular apply or a diff, Kubernetes will throw an Error from server (NotFound): namespaces "xxxxxxxx" not found error.

This is because Kubernetes can't apply a namespace scoped resources on a missing namespace (we use server-side apply on the diff too), Kubernetes handles a server-side dry-run as a regular apply except that it doesn't store the final resources (it triggers webhooks too).

More info here: kubernetes/kubernetes#83562

Helm solves this by creating the namespaces beforehand, in Helm2 this was the default behaviour and was very annoying. In Helm3 they made a better choice and make this non-default optional with --create-namespace flag. We could do the same.