Code Monkey home page Code Monkey logo

Comments (2)

gjenkins8 avatar gjenkins8 commented on September 25, 2024

I suspect (guess) that the job had a different condition that Helm didn't know about. Can you show the status output of e.g. kubectl describe job, both before and after the job finished being "active" please.

from helm.

polasekr avatar polasekr commented on September 25, 2024

I am experiencing the same issue with Kyverno chart, where it is failing to upgrade due to incorrect job state detection.

From the debug output I can see


helm:QPQQU> client.go:740: [debug] Add/Modify event for infra-kyverno-clean-reports: MODIFIED
helm:QPQQU> client.go:779: [debug] infra-kyverno-clean-reports: Jobs active: 0, jobs failed: 0, jobs succeeded: 0
helm:QPQQU> 
helm:QPQQU> client.go:486: [debug] Starting delete for "infra-kyverno-clean-reports" Job
helm:QPQQU> 
helm:QPQQU> wait.go:104: [debug] beginning wait for 1 resources to be deleted with timeout of 5m0s
helm:QPQQU> 
helm:QPQQU> upgrade.go:476: [debug] warning: Upgrade "infra" failed: post-upgrade hooks failed: 1 error occurred:
helm:QPQQU> * timed out waiting for the condition

and when I run kubectl describe job

$ kubectl describe job infra-kyverno-clean-reports 
Name:             infra-kyverno-clean-reports
Namespace:        infra
Selector:         batch.kubernetes.io/controller-uid=78201865-0e6c-4c44-92a6-a2b4c856fb73
Labels:           app.kubernetes.io/component=hooks
                  app.kubernetes.io/instance=infra
                  app.kubernetes.io/managed-by=Helm
                  app.kubernetes.io/part-of=infra-kyverno
                  app.kubernetes.io/version=3.2.5
                  helm.sh/chart=kyverno-3.2.5
Annotations:      helm.sh/hook: post-upgrade
                  helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded,hook-failed
Parallelism:      1
Completions:      1
Completion Mode:  NonIndexed
Start Time:       Tue, 13 Aug 2024 12:23:50 -0400
Completed At:     Tue, 13 Aug 2024 12:23:57 -0400
Duration:         7s
Pods Statuses:    0 Active (0 Ready) / 1 Succeeded / 0 Failed
Pod Template:
  Labels:           batch.kubernetes.io/controller-uid=78201865-0e6c-4c44-92a6-a2b4c856fb73
                    batch.kubernetes.io/job-name=infra-kyverno-clean-reports
                    controller-uid=78201865-0e6c-4c44-92a6-a2b4c856fb73
                    job-name=infra-kyverno-clean-reports
  Service Account:  kyverno-admission-controller
  Containers:
   kubectl:
    Image:           bitnami/kubectl:1.28.5
    Port:            <none>
    Host Port:       <none>
    SeccompProfile:  RuntimeDefault
    Command:
      /bin/bash
      -c
      set -euo pipefail
      NAMESPACES=$(kubectl get namespaces --no-headers=true | awk '{print $1}')
      
      for ns in ${NAMESPACES[@]};
      do
        COUNT=$(kubectl get policyreports.wgpolicyk8s.io -n $ns --no-headers=true | awk '/pol/{print $1}' | wc -l)
      
        if [ $COUNT -gt 0 ]; then
          echo "deleting $COUNT policyreports in namespace $ns"
          kubectl get policyreports.wgpolicyk8s.io -n $ns --no-headers=true | awk '/pol/{print $1}' | xargs kubectl delete -n $ns policyreports.wgpolicyk8s.io
        else
          echo "no policyreports in namespace $ns"
        fi
      done
      
      COUNT=$(kubectl get clusterpolicyreports.wgpolicyk8s.io --no-headers=true | awk '/pol/{print $1}' | wc -l)
        
      if [ $COUNT -gt 0 ]; then
        echo "deleting $COUNT clusterpolicyreports"
        kubectl get clusterpolicyreports.wgpolicyk8s.io --no-headers=true | awk '/pol/{print $1}' | xargs kubectl delete clusterpolicyreports.wgpolicyk8s.io
      else
        echo "no clusterpolicyreports"
      fi
      
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Events:
  Type    Reason            Age    From            Message
  ----    ------            ----   ----            -------
  Normal  SuccessfulCreate  3m29s  job-controller  Created pod: infra-kyverno-clean-reports-cz5fj
  Normal  Completed         3m22s  job-controller  Job completed

It seems to be some bug.

Versions:

Kubernetes
Client Version: v1.27.1
Kustomize Version: v5.0.1
Server Version: v1.29.3

helm version v3.15.3

from helm.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.