Investigate Helm not updating deployments correctly about armada HOT 2 CLOSED

well i'm having the same issue with helm version 3.0.2
using configMaps to inject a script in a post-install hook container to configure the application
for environment specific settings.

while testing & fine tuning the script , i noticed that upgrade doesn't touch the configmap and doesn't change/update it :-(

a debug trace :

• helm -n cornerstone upgrade -f ../global.yaml --debug cornerstone .
  upgrade.go:79: [debug] preparing upgrade for cornerstone
  upgrade.go:87: [debug] performing update for cornerstone
  upgrade.go:225: [debug] creating upgraded release for cornerstone
  client.go:222: [debug] Starting delete for "job-setup-cornerstone" Job
  client.go:89: [debug] creating 1 resource(s)
  client.go:422: [debug] Watching for changes to Job job-setup-cornerstone with timeout of 5m0s
  client.go:450: [debug] Add/Modify event for job-setup-cornerstone: ADDED
  client.go:489: [debug] job-setup-cornerstone: Jobs active: 0, jobs failed: 0, jobs succeeded: 0
  client.go:450: [debug] Add/Modify event for job-setup-cornerstone: MODIFIED
  client.go:489: [debug] job-setup-cornerstone: Jobs active: 1, jobs failed: 0, jobs succeeded: 0
  client.go:450: [debug] Add/Modify event for job-setup-cornerstone: MODIFIED
  upgrade.go:280: [debug] warning: Upgrade "cornerstone" failed: pre-upgrade hooks failed: job failed: BackoffLimitExceeded

FYI : the current scripts ends with exit 1 for debugging purpose, until we have validated the upgrade script .

the script is inline in the configmap

from armada.

Closing this as:

• We don't deploy directly from this repo anymore (which is what we originally created the issue for)

We have not seen this occur on any of our deployments for a while. We now rely on:

• We now use secrets to hold our secrets (as some of the config is secret)
• We now use checksum annotations in our deployments:
  • https://github.com/G-Research/armada/blob/master/deployment/armada/templates/deployment.yaml#L20
  • https://github.com/G-Research/armada/blob/master/deployment/executor/templates/deployment.yaml#L16

That is not to say this issue no longer exists in helm, just that we haven't experienced any issues with this setup.

from armada.