Currently there is no(?) support to include the updated image/tag in the commit message.
This does not provide a very useful commit log when the automation is used.
The default, "Update from image update automation" should also include the resource being updated and to which image or tag it has updated to.

Expected (suggestion):
For an HelmRelease: The default commit message should be in the form of " updated to ", e.g "service1 updated to 1.2.3-a6b8c6c563b5f3db2ceb1ee9d548acd2abb6baaf-12345"

Should be supported in both the default commit message as well as in a custom messageTemplate

Desired:
Possible to configure a global commit message template for the entire cluster

Image automation controller gives the following error log:

{
  "level": "error",
  "ts": "2021-01-27T12:34:46.613Z",
  "logger": "controller-runtime.manager.controller.imageupdateautomation",
  "msg": "Reconciler error",
  "reconciler group": "image.toolkit.fluxcd.io",
  "reconciler kind": "ImageUpdateAutomation",
  "name": "flux-system",
  "namespace": "flux-system",
  "error": "unknown error: remote: ",
  "stacktrace": "sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:248\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:211\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99"
}

I am using gitlab token to provision the flux deployment. The command that I use as follows:

export GITLAB_TOKEN="xxxx"

flux bootstrap gitlab \
    --owner=xxx/xxx \
    --repository="xxx"  \
    --branch=master \
    --log-level debug \
    --hostname="gitlab.com" \
    --path="xxx/xxx" \
    --components-extra=image-reflector-controller,image-automation-controller

Flux version:

flux version 0.7.2

With the error log that I receive from image-automation-controller, I do not see useful information to troubleshoot this issue.

Hi all,

First of all, thanks for the work.

I am using the latest version of Flux2 and image-automation-controller / reflector. Given the following resources:

---
# Source: environment/templates/env-sync.yaml
apiVersion: source.toolkit.fluxcd.io/v1beta1
kind: GitRepository
metadata:
  name: dev-123
  namespace: flux-system
spec:
  interval: 1m0s
  ref:
    branch: master
  secretRef:
    name: flux-system
  url: https://gitlab.com/unodos/gitlab-ci/gitops
  ignore: |
    # exclude all but environment dev-123 patches.yaml
    /*
    !/cluster/environments/dev-123/patches.yaml
---
# Source: environment/templates/patches.yaml
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: podinfo
  namespace: dev-123
spec:
  values:
    image:
      tag: 5.0.1 # {"$imagepolicy": "flux-system:dev-123-podinfo-policy:tag"}
# Source: environment/templates/image-automations.yaml
apiVersion: image.toolkit.fluxcd.io/v1alpha1
kind: ImageUpdateAutomation
metadata:
  name: dev-123-podinfo-automation
  namespace: flux-system
spec:
  interval: 1m
  checkout:
    branch: master
    gitRepositoryRef:
      name: dev-123
  update:
    setters: {}
  commit:
    authorName: UpdateBot
    authorEmail: [email protected]

I am seeing this error:

image-automation-controller-5849df9999-tftj5 manager {"level":"error","ts":"2020-12-21T13:18:30.688Z","logger":"controller","msg":"Reconciler error","reconcilerGroup":"image.toolkit.fluxcd.io","reconcilerKind":"ImageUpdateAutomation","controller":"imageupdateautomation","name":"dev-123-podinfo-automation","namespace":"flux-system","error":"/tmp/flux-system-dev-123935373182/helm/charts/environment/templates/image-automations.yaml: yaml: line 3: did not find expected key"}

Note that the path is out of scope for the Gitrepository CRD I have created (only includes cluster/environments/dev-123/patches.yaml). Is this an expected behavior?

Thanks!

.. so you can create an ImageUpdateAutomation with flux.

There's a flag (default true 😢 ) for watching all namespaces, vs watching just the namespace the controller is deployed to.

For example, see

• https://github.com/fluxcd/kustomize-controller/blob/main/main.go#L96
• https://github.com/fluxcd/kustomize-controller/blob/main/main.go#L101

Sometimes the image ref or tag (or digest) is part of a longer string in a field value. For example, in a container argument:

kind: Pod
spec:
  containers:
  - name: hello
    image: helloworld
    args:
    - --tag-argument=1.9.0 # <-- here

Using a setter won't work, since it'll replace the whole value. What's needed is a way to give a template for the whole value.

The existing solution to this in kyaml is "substitutions": you define a substitution in your schema file, which has a template with references to setter names; then, you name the substitution alongside the field. This is tricky in image-update-automation for a couple of reasons:

• there's no schema file in which to put substitutions; possibly they could go in the automation, but this is distant to the point at which it is applied, so seems like the wrong place;
• with simple setters, any that aren't defined (because they aren't in the namespace, or don't exist) will be ignored. How do you restrict what can be referred to in the template?

For these reasons, I propose a scheme like this:

• the template is given in the marker
• image policies are named in the same format as for simple setters; anything that refers to policies in different namespaces is ignored

This will mean implementing a kyaml filter, since the kyaml setters2 code won't handle templates given in the marker.

• watch image policy objects (that are mentioned in automations)
• possibly watch git repository objects -- they will signal when new commits are pulled (assuming you're running the source controller, and it's in the right namespace)
• .. but also run periodically

It should also be required, rather than optional. This to harmonise with the other GOTK controllers.

• some characterisation of the reason for the update
• the name of the automation object that triggered it
• the list of objects that were changed

The diff will already say what changed; the requirement here is to say why it changed, and give any other details that may be pertinent.

GOTK has a notification-controller which can be used to forward events from the controllers to external systems, e.g., Slack, or webhooks. The automation controller has an event which is worthy of a notification, which is when it's successfully made a commit.

I have the latest Flux2 tooling installed but the image-automation-controller throws an error on startup because of missing GitRepository in version source.toolkit.fluxcd.io/v1alpha1.

If I look at my GitRepositories they are in version source.toolkit.fluxcd.io/v1beta1.
So for me it seems to be a compatibility issue.

2020-11-20T12:23:26.929Z    INFO    controller-runtime.metrics    metrics server is starting to listen    {"addr":"127.0.0.1:8080"}
2020-11-20T12:23:26.930Z    INFO    setup    starting manager
I1120 12:23:26.930423       6 leaderelection.go:242] attempting to acquire leader lease  flux-system/79628f79.fluxcd.io...
2020-11-20T12:23:26.930Z    INFO    controller-runtime.manager    starting metrics server    {"path": "/metrics"}
I1120 12:23:42.448479       6 leaderelection.go:252] successfully acquired lease flux-system/79628f79.fluxcd.io
2020-11-20T12:23:42.449Z    INFO    controller    Starting EventSource    {"reconcilerGroup": "image.toolkit.fluxcd.io", "reconcilerKind": "ImageUpdateAutomation", "controller": "imageupdateautomation", "source": "kind source: /, Kind="}
2020-11-20T12:23:42.449Z    INFO    controller    Starting EventSource    {"reconcilerGroup": "image.toolkit.fluxcd.io", "reconcilerKind": "ImageUpdateAutomation", "controller": "imageupdateautomation", "source": "kind source: /, Kind="}
2020-11-20T12:23:42.448Z    DEBUG    controller-runtime.manager.events    Normal    {"object": {"kind":"ConfigMap","namespace":"flux-system","name":"79628f79.fluxcd.io","uid":"ce44514b-fd42-4f42-b9c4-9dbd89dd3ca1","apiVersion":"v1","resourceVersion":"7514737"}, "reason": "LeaderElection", "message": "image-automation-controller-5c556d544-2gtxg_28233d8b-7097-4625-b62c-7979c5f07e7c became leader"}
2020-11-20T12:23:43.053Z    ERROR    controller-runtime.source    if kind is a CRD, it should be installed before calling Start    {"kind": "GitRepository.source.toolkit.fluxcd.io", "error": "no matches for kind \"GitRepository\" in version \"source.toolkit.fluxcd.io/v1alpha1\""}

The general pattern within GitOps Toolkit is that resources can be suspended (don't act on it), and those running on a schedule can be run on demand (act on it now). See, for example, https://toolkit.fluxcd.io/components/kustomize/kustomization/#reconciliation

(A recent example of doing this is fluxcd/image-reflector-controller#27)

Hello! Thanks so much for making a more flexible and more secure version--so far it is taking some getting used to, but I am liking what I am seeing!

One issue I am not sure if you are planning for: our workflow is to keep our kustomize manifests in one directory, and then build those into single yaml files that live in another directory, and configure flux v1 to look at that directory. We have some pretty complex patches and this makes it easy to see exactly what is getting deployed into the cluster.

Because kustomize does not keep comments intact when running kustomize build, we are having some issues with the image update markers. We either have to annotate the Kustomization that flux created, or go in after kustomize has built the yaml and modify it there (a pain, especially for some of our automation). So far, image automation has worked on all Deployments because we can define the image policy marker comment in the Kustomization object that Flux creates:

apiVersion: kustomize.toolkit.fluxcd.io/v1beta1
kind: Kustomization
metadata:
  name: flux-system
  namespace: flux-system
spec:
  interval: 10m0s
  path: ./deploy/dev
  prune: true
  sourceRef:
    kind: GitRepository
    name: flux-system
  validation: client
  images:
  - name: someRepo/someImage
    newName: someRepo/someImage
    newTag: 1.0.2 # {"$imagepolicy": "flux-system:test-policy:tag"}

However, this does not work for HelmRelease objects, which makes sense because often times image and tag are split up. An example HelmRelease might be:

apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: nginx-test
spec:
  chart:
    spec:
      chart: charts/nginx
      sourceRef:
        kind: GitRepository
        name: gitops-helm
        namespace: flux-system
      version: 0.1.0
  interval: 5m
  values:
    image:
      repository: someOtherRepo/someOtherImage
      tag: 0.1.0

Previously in flux v1, we could just annotate the HelmRelease object and call it a day, but since annotation-based image updates are gone, we need to find a way to do it here. I was hoping that there would be a similar path as above and I could add something like this to the flux Kustomization and have it respect the path for any workloads with that image, but they are not getting updated:

 images:
  - name: someOtherRepo/someOtherImage
    newName: someOtherRepo/someOtherImage
    newTag: 1.0.2 # {"$imagepolicy": "flux-system:test-policy:image:tag"}

I know this is achievable by using a Kustomization object and pointing it at the folder that contains the manifests, but that defeats the purpose of having the built file be what is deployed into the cluster.

Is there a solution that would allow us to build our kustomize directories into complete yamls and still get automated image updates? I am really hoping something like this is possible or on the roadmap, because it was quite simple in flux v1.

Thanks!

Both the standard GOTK metrics, if they make sense, and any metrics particular to the automation.

The controller should at least use a Ready condition, even if its interpretation will be a little interesting. (Perhaps check what CronJob resources have for conditions).

Similarly, it probably needs more careful status updates. Look at recording the revision last pushed, that sort of thing.

When replacing images in a workload, it'd be good to keep the format that the workload itself uses, and just replace the tag (or digest, if that's supported). E.g., if the workload uses

image: helloworld:3.8

and the policy has

status:
  latestImage: index.docker.io/library/helloworld:3.9

then the update should be to

image: helloworld:3.9

(I'll have to look at examples for this)

If no branch is specified in the image automation object, instead of using the branch set in the referenced GitRepository, the controller uses master and this breaks the push. We should make the branch required or use the one specified in the GitRepository.

HI All,

I checked to see if this had been asked, but I am trying to use the below inside a GKE cluster within the same project:

apiVersion: image.toolkit.fluxcd.io/v1alpha1
kind: ImageRepository
metadata:
  name: hello-world
  namespace: flux-system
spec:
  url: gcr.io/<PROJECT_ID>
  image: hello-world
  interval: 1m0s

however when I run flux get image repository I get

hello-world	False	GET https://gcr.io/v2/<PROJECT_ID>/hello-world/tags/list?n=1000: DENIED: Failed to read tags for host 'gcr.io', repository '/v2/<PROJECT_ID>//hello-world/tags/list?n=1000'

I have confirmed that the node service account can pull images, and I've also confirmed that it can list tags for this image using that service account identity.

I noticed that GCR support is listed as TODO, but I've seen some code snippets that lead me to believe this should work. Any advice? Anything I can do to help? GCR support would be super helpful.

controller-runtime has helpers for this: see for example https://github.com/fluxcd/kustomize-controller/blob/8bb4f4c80b4d257c32e2a36349d9687de966af0d/main.go#L158

We use an ssh-deploy key to have flux talk to repos (we use gitlab.com)

When we don't enable write-access on the git-repo for the deploy-key, flux (correctly) fails to write the image updates, but does not provide a nice error message as to why.

We get the following: (this is also output when using flux get image update)

message: 'unknown error: remote: '

apiVersion: v1
items:
- apiVersion: image.toolkit.fluxcd.io/v1alpha1
  kind: ImageUpdateAutomation
  metadata:
    annotations:
    generation: 2
    labels:
      kustomize.toolkit.fluxcd.io/checksum: 0a827fca90af236d1cb0d0e2c757681441c6ccfa
      kustomize.toolkit.fluxcd.io/name: flux-system
      kustomize.toolkit.fluxcd.io/namespace: flux-system
    name: <redacted>
    namespace: flux-system
  spec:
    checkout:
      branch: test-flux2-image-updates
      gitRepositoryRef:
        name:<redacted>
    commit:
      authorEmail: [email protected]
      authorName: fluxcdbot
      messageTemplate: '[ci skip] update image'
    interval: 1m0s
    update:
      setters: {}
  status:
    conditions:
    - lastTransitionTime: "2021-01-28T14:01:47Z"
      message: 'unknown error: remote: '
      reason: ReconciliationFailed
      status: "False"
      type: Ready
    observedGeneration: 2

https://toolkit.fluxcd.io/guides/image-update/:
ImageUpdateAutomation takes a branch parameter, but so does the git-repo it refers, this seems ambiguous to me, you want to refer the git repo it was defined in, and that's already defined in the GitRepository - hence it should be sufficient with the gitRepositoryRef.

spec:
checkout:
branch: main <--
gitRepositoryRef:
name: flux-system <-- this repo already has a branch defined

Maybe a bit related to #85

Now on source-controller we can create a gitrepository resource for azure using implementation libgit2. Related bug (closed) is fluxcd/source-controller#104

Example:

flux create source git flux-system \
  --git-implementation=libgit2 \
  --secret-ref=git-ssh-key \
  --url=ssh://[email protected]/v3/org/collection/my_flux_repository \
  --branch=main \
  --interval=1m

When the image-automation-controller try to get from git, it fails since it's not following the git-implementation directive.

It would be very nice to be able to have the full flow working. (Adding flux -to- auto-update)

Right now the error is as below:

{"level":"error","ts":"2020-12-16T18:12:00.877Z","logger":"controller","msg":"Reconciler error","reconcilerGroup":"image.toolkit.fluxcd.io","reconcilerKind":"ImageUpdateAutomation","controller":"imageupdateautomation","name":"flux-system","namespace":"flux-system","error":"unable to clone 'ssh://[email protected]/v3/org/collection/my_flux_repository', error: empty git-upload-pack given"}

Images:
source-controller:v0.5.4
image-automation-controller:v0.1.0
image-reflector-controller:v0.1.0

Even if the automation runs fine and commits are being pushed to Git, the status show "no updates made".

Sometimes people want to use the image digest rather than (or as well as?) the tag. Assuming fluxcd/image-reflector-controller#87, this controller could add extra setters for the digest (e.g., sha256:abc123... of named policies (and the image ref with digest, e.g., alpine@sha256:abc123...).

Not sure how relevant these are, since envtest is pretty viable. But at least think a bit about it ...

It's the convention for gitops-toolkit.

Hi,
Following the suggestion on slack I took a look to this repository to experiment a bit with the image-automation-controller and, maybe in the future, contribute to this really interesting project.
Once I've cloned the repository I navigated to /config/default folder and I launched the following command:

kustomize build | kubectl apply -f-

Checking the status of the image-automation-controller I saw that is in status ImagePullBackOff.
Looking for the events, I go these ones:

  Normal   Pulled     80s                kubelet            Container image "gcr.io/kubebuilder/kube-rbac-proxy:v0.5.0" already present on machine
  Normal   Created    80s                kubelet            Created container kube-rbac-proxy
  Normal   Started    80s                kubelet            Started container kube-rbac-proxy
  Normal   Pulling    40s (x3 over 80s)  kubelet            Pulling image "fluxcd/image-automation-controller"
  Warning  Failed     39s (x3 over 79s)  kubelet            Failed to pull image "fluxcd/image-automation-controller": rpc error: code = Unknown desc = Error response from daemon: manifest for fluxcd/image-automation-controller:latest not found: manifest unknown: manifest unknown
  Warning  Failed     39s (x3 over 79s)  kubelet            Error: ErrImagePull
  Normal   BackOff    6s (x5 over 78s)   kubelet            Back-off pulling image "fluxcd/image-automation-controller"
  Warning  Failed     6s (x5 over 78s)   kubelet            Error: ImagePullBackOff

After a quick search inside the fluxcd repository I saw that no container is tagged as latest and this is causing the error, the only tag present is: v0.0.1-alpha.1.
This issue can easily be fixed by creating a patch in the the config folder by specifying which container use but since I'm not sure how to do that I opened this issue.

Hi there,

I can see there was an "path" field in the spec which was discussed in #24 and removed with pull request #64 due to missing use-cases. I might have a possible use-case. The flux2 documentation states how to setup up multiple clusters/stages in different folders of the same repository. Like the folder "dev" is deployed into the dev-cluster, the folder "prod" to the prod-cluster an so on. But this can cause conflicts. If different clusters have ImagePolicies with the same name, but different spec, the automation-controller of the dev-cluster will not only adjust the version-tags in the "dev" folder, but also in the folders of the other stages. Possible solutions would be:

1. to use different branches for each stage, or even dedicated repositories but this would limit the use of a "base-kustomization" folder which is reused in all stages
2. to use different policy names in all stages. But there is still the eventuality that a developer accidentally uses a policy name which is already used in an other stage and breaks something.

To implement the "path" field would be a big improvement in this case.

Best regards
Philipp

This is an issue to raise awareness that kustomize kyaml has some serious bugs that could affect Flux users:

• If you have non-ASCII characters in your YAMLs the controller will panic and crash loop kubernetes-sigs/kustomize#3605
• kyaml: fatal error: concurrent map read and map write kubernetes-sigs/kustomize#3659
• If you have non-string keys in your manifests, the image-automation-controller will not be able to patch the image tags, ref: kubernetes-sigs/kustomize#3446
• YAML anchors and aliases return error on List types kubernetes-sigs/kustomize#3614
• If you have duplicate keys in your manifests, the image-automation-controller will crash loop, ref: kubernetes-sigs/kustomize#3659 (fixed)

Since image-automation-controller depends on kyaml setters, we can't just disable it like we did for kustomize-controller.

See https://github.com/fluxcd/image-reflector-controller/pull/33/files for an example.

I have the following setup:

Flux2 is running in namespace flux-system.
Prometheus is running in namespace monitoring.

I am trying to get image automation to work with as little duplication as possible.

So, I defined ImagePolicy (and ImageRepository) close to Prometheus. This means, because I use kustomize, that it is forced in the monitoring namespace.
Next, I defined ImageUpdateAutomation in the flux-system namespace, as it wants access to the GitRepository which is already defined there.

Now my problem: this setup doesn't work when I try # {"$imagepolicy": "monitoring:prometheus:tag"}.

From what I understand, and what does work:

• either ImagePolicy needs to be in the flux-system namespace (which is far from ideal from a Infrastructure-as-Code point of view)
• or I need to move ImageUpdateAutomation into monitoring namespace. I don't really have an issue with that, except for the minor code duplication, but the gitRepositoryRef doesn't accept namespace. That would mean I also have to duplicate GitRepository, which would be very prune to errors.

Now I am not sure that what I am trying here is how this is meant to be used, or that I am overlooking some setting somewhere. So if you have any pointers for me how I should be doing this, or if there is an option I can use to make this work, that would be very welcome :)

Thank you!

The path to find image references is hard-wired to .spec.template.spec.{containers,initContainers}[*].image (in effect; YAML path is not actually used). At least CronJobs have a different structure, so will fail to be updated.

The simplest thing to accomodate CronJob would be to dispatch on the Kind, using a different path-traversing func.

To update HelmRelease will need a deeper change; I'll have to look at the code in flux and see if I can adapt that to kyaml.

(not our own apiVersion, but any types used from e.g., source.toolkit.fluxcd.io)

Use the logger setup from https://github.com/fluxcd/pkg/tree/main/runtime/logger and review use of logging vs dev guide / examples elsewhere.

It seems like the image is stored as a label, and that this labels value is too large? Maybe use an annotation for the actual value?

der    False   Helm upgrade failed: cannot patch "ghec-onboarder-fsjetty" with kind ServiceAccount: ServiceAccount "ghec-onboarder-fsjetty" is invalid: [metadata.labels: Invalid value: "fsnexus.evry.com:8085/evryfs/ccm/ccm-userclean-ghec-onboarder:2.0.212": must be no more than 63 characters, metadata.labels: Invalid value: "fsnexus.evry.com:8085/evryfs/ccm/ccm-userclean-ghec-onboarder:2.0.212": a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue',  or 'my_value',  or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')] && cannot patch "ghec-onboarder-fsjetty-imagepullsecret" with kind Secret: Secret "ghec-onboarder-fsjetty-imagepullsecret" is invalid: [metadata.labels: Invalid value: "fsnexus.evry.com:8085/evryfs/ccm/ccm-userclean-ghec-onboarder:2.0.212": must be no more than 63 characters, metadata.labels: Invalid value: "fsnexus.evry.com:8085/evryfs/ccm/ccm-userclean-ghec-onboarder:2.0.212": a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue',  or 'my_value',  or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')] && cannot patch "ghec-onboarder-fsjetty-config" with kind ConfigMap: ConfigMap "ghec-onboarder-fsjetty-config" is invalid: [metadata.labels: Invalid value: "fsnexus.evry.com:8085/evryfs/ccm/ccm-userclean-ghec-onboarder:2.0.212": must be no more than 63 characters, metadata.labels: Invalid value: "fsnexus.evry.com:8085/evryfs/ccm/ccm-userclean-ghec-onboarder:2.0.212": a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue',  or 'my_value',  or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')] && cannot patch "ghec-onboarder-fsjetty" with kind Role: Role.rbac.authorization.k8s.io "ghec-onboarder-fsjetty" is invalid: [metadata.labels: Invalid value: "fsnexus.evry.com:8085/evryfs/ccm/ccm-userclean-ghec-onboarder:2.0.212": must be no more than 63 characters, metadata.labels: Invalid value: "fsnexus.evry.com:8085/evryfs/ccm/ccm-userclean-ghec-onboarder:2.0.212": a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue',  or 'my_value',  or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')] && cannot patch "ghec-onboarder-fsjetty" with kind RoleBinding: RoleBinding.rbac.authorization.k8s.io "ghec-onboarder-fsjetty" is invalid: [metadata.labels: Invalid value: "fsnexus.evry.com:8085/evryfs/ccm/ccm-userclean-ghec-onboarder:2.0.212": must be no more than 63 characters, metadata.labels: Invalid value: "fsnexus.evry.com:8085/evryfs/ccm/ccm-userclean-ghec-onboarder:2.0.212": a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue',  or 'my_value',  or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')] && cannot patch "ghec-onboarder-fsjetty" with kind Service: Service "ghec-onboarder-fsjetty" is invalid: [metadata.labels: Invalid value: "fsnexus.evry.com:8085/evryfs/ccm/ccm-userclean-ghec-onboarder:2.0.212": must be no more than 63 characters, metadata.labels: Invalid value: "fsnexus.evry.com:8085/evryfs/ccm/ccm-userclean-ghec-onboarder:2.0.212": a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue',  or 'my_value',  or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')] && cannot patch "ghec-onboarder-fsjetty" with kind Deployment: Deployment.apps "ghec-onboarder-fsjetty" is invalid: [metadata.labels: Invalid value: "fsnexus.evry.com:8085/evryfs/ccm/ccm-userclean-ghec-onboarder:2.0.212": must be no more than 63 characters, metadata.labels: Invalid value: "fsnexus.evry.com:8085/evryfs/ccm/ccm-userclean-ghec-onboarder:2.0.212": a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue',  or 'my_value',  or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')]   152m

Some people like to keep their configuration alongside their application code, and set up CI to build images from the application code. This can cause problems when combined with Flux's automation, because commits from the automation will cause the CI to run (which builds another image, which causes the automation to run, ...).

Happily, many (most?) CI systems have a "get out clause": if you put a special token in a commit message, they will not run for that commit. For example, Circle CI uses [ci skip]: https://circleci.com/docs/2.0/skip-build/ (and someone summarised the support in a whole lot of systems here: https://github.community/t/github-actions-does-not-respect-skip-ci/17325/3).

By including this token in the default commit message (which people can change if they don't want to skip CI for automated commits), the looping problem above will be avoided in a default install.

EDIT: expanded description.

It would be quite handy to be able to use a tool to mark all the uses of an image, rather than having to go and add the markers yourself.

I want to use the image-policy replacement mechanism on a literal-value inside my config-map generator of kustomize.

configMapGenerator:
- name: frontend
  literals:
  - NGINX_PORT=80
  - APP_VERSION=1.9.0 # {"$imagepolicy": "flux-system:webapp-dev:tag"}

The flux-bot makes a wrong replacement like this. The key of the literal gets lost.

configMapGenerator:
- name: frontend
  literals:
  - NGINX_PORT=80
  - 1.9.0 # {"$imagepolicy": "flux-system:webapp-dev:tag"}

Example: fluxcd/image-reflector-controller#21 (mainly first commit; check others)

I was fiddling with image automation a bit, and I managed to hit this error:

{"level":"error","ts":"2021-01-13T15:57:20.289Z","logger":"controller","msg":"Reconciler error","reconcilerGroup":"image.toolkit.fluxcd.io","reconcilerKind":"ImageUpdateAutomation","controller":"imageupdateautomation","name":"flux-system","namespace":"flux-system","error":"unknown error: ERROR: The key you are authenticating with has been marked as read only."}

Which totally makes sense, as the GitHub key I use is set to read-only.

What I did notice how-ever, is that this error is not shown to me on Discord, with an unmodified flux2 setup via Kustomize (configured to display errors on Discord). I would have expected this error to end up there, as most other errors flux2 generates does.

When I looked further into this, I noticed that eventSources of Alert did not read ImageUpdateAutomation yet, but .. this is also not supported (yet?).

The update field of the ImageUpdateAutomation spec can only contain one value, since paths was removed, and that's { setters: {} }. It isn't defaulted in the schema or in code, so you need to provide

# ...
spec:
  update:
    setters: {}

which is ugly.

Using kubebuilder:default to default it to { setters: {} } doesn't work, because YAML (I think) interprets that inner value as null, and the schema won't accept null.

Assuming we want to keep room for other strategies, a nicer representation might be to use an enum, and (embedded) fields for any parameters:

spec:
  update:
    strategy: Setters
    # any strategy parameters go here, either embedded or in an object

which means the default value for update can be { strategy: "Setters" }, which is more pleasing.

I did this for image-reflector-controller in fluxcd/image-reflector-controller#33

Currently the imageautomationupdate I created is failing with the error: ** error: user cancelled hostkey check**

The system is setup as follows:

---
apiVersion: source.toolkit.fluxcd.io/v1beta1
kind: GitRepository
metadata:
  name: base-backend-repo
  namespace: tenant-backend
spec:
  interval: 2m0s
  ref:
    branch: main
  url: ssh://[email protected]/v3/XXX/XXX/XXX
  gitImplementation: libgit2 # Special for azure devops
  secretRef:
    name: ssh-credentials
  ignore: |
    .git/
    .github/
    scripts/

apiVersion: image.toolkit.fluxcd.io/v1alpha1
kind: ImageUpdateAutomation
metadata:
  name: tenant-backend-image-update
  namespace: tenant-backend
spec:
  checkout:
    branch: main
    gitRepositoryRef:
      name: base-backend-repo
  commit:
    authorEmail: [email protected]
    authorName: fluxcdbot
    messageTemplate: '[ci skip] update image'
  interval: 1m0s
  update:
    strategy: Setters

This results in the following:

│   Conditions:                                                                                                                                                                                                                                                                                                                                                  │
│     Last Transition Time:  2021-02-05T17:27:30Z                                                                                                                                                                                                                                                                                                                │
│     Message:               unable to clone 'ssh://[email protected]/v3/XXX/XXX/XXX', error: user cancelled hostkey check                                                                                                                                                                                                │
│     Reason:                ReconciliationFailed                                                                                                                                                                                                                                                                                                                │
│     Status:                False                                                                                                                                                                                                                                                                                                                               │
│     Type:                  Ready                                                                                                                                                                                                                                                                                                                               │
│   Observed Generation:     1                                                                                                                                                                                                                                                                                                                                   │
│ Events:                                                                                                                                                                                                                                                                                                                                                        │
│   Type    Reason  Age                      From                         Message                                                                                                                                                                                                                                                                                │
│   ----    ------  ----                     ----                         -------                                                                                                                                                                                                                                                                                │
│   Normal  error   4m21s (x334 over 3d15h)  image-automation-controller  unable to clone 'ssh://[email protected]/v3/XXX/XXX/XXX', error: user cancelled hostkey check

The ssh credentials are correct, because all other processes are working as expected, so there seems to be only an issue in the image update controller.

Example: https://github.com/fluxcd/kustomize-controller/tree/main/docs/spec

Describe the bug

Blank lines are removed from my kustomization.yaml after an image update. In large kustomizations it useful to seperate sections for readability.

To Reproduce

Start with a kustomization.yaml like so:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: production-my-app

namePrefix: production-

commonLabels:
  overlay: production

resources:
- namespace.yaml
- ../base

images:
- name: ghcr.io/stefanprodan/podinfo
  newTag: 5.0.0 # {"$imagepolicy": "flux-system:my-app-production:tag"}

After an image update, the blank lines are removed :(

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: production-my-app
namePrefix: production-
commonLabels:
  overlay: production
resources:
- namespace.yaml
- ../base
images:
- name: ghcr.io/stefanprodan/podinfo
  newTag: 5.1.4 # {"$imagepolicy": "flux-system:my-app-production:tag"}

Expected behavior

Maintain the blank lines.

Additional context

flux version 0.9.0

In addition to docs generated from the API types, by convention there's a more user-oriented reference to the API types. See for example https://toolkit.fluxcd.io/components/helm/helmreleases/ (as opposed to the generated doc, https://github.com/fluxcd/source-controller/blob/main/docs/spec/v1beta1/helmrepositories.md.

Include the image-reflector types as well. This all goes in https://github.com/fluxcd/flux2/tree/main/docs/guides/

Hi 👋

Thank you for your awesome job. I've tested it yesterday and it works like a charm. I've noticed a strange behavior when running automation: every yaml of my repository was rewritten with another formatting.

For exemple, the first commit of image-automation-controller on my repository is this one. The version is correctly updated but why the rest also?

I have specific rules for yaml formatting (mainly relying on prettier), is there a way (a parameter, a tag) to limit updates to only the lines tagged or only the files which contain tags?

The ImageUpdateAutomationSpec includes a paths field, with the idea being that updates are only applied to files within the paths given.