Code Monkey home page Code Monkey logo

Comments (4)

jessesuen avatar jessesuen commented on August 28, 2024

Simplified rollout object:

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  annotations:
    rollout.argoproj.io/revision: '16'
  generation: 1
  labels:
    app: partner-support-portal
  name: partner-support-portal
spec:
  minReadySeconds: 30
  paused: false
  replicas: 2
  selector:
    matchLabels:
      app: partner-support-portal
  strategy:
    blueGreen:
      activeService: partner-support-portal-active
      previewService: partner-support-portal-preview
  template:
    metadata:
      labels:
        app: partner-support-portal
    spec:
      containers:
        - image: someimage:sometag
          name: partner-support-portal
status:
  HPAReplicas: 2
  availableReplicas: 5
  blueGreen:
    activeSelector: 5f59c4c4dc
    previewSelector: 7c67b5fd8d
  canary: {}
  conditions:
    - lastTransitionTime: '2019-05-20T07:22:41Z'
      lastUpdateTime: '2019-05-20T07:22:41Z'
      message: Rollout has minimum availability
      reason: AvailableReason
      status: 'True'
      type: Available
    - lastTransitionTime: '2019-05-20T11:02:00Z'
      lastUpdateTime: '2019-05-20T11:02:00Z'
      message: Rollout "partner-support-portal" has timed out progressing.
      reason: ProgressDeadlineExceeded
      status: 'False'
      type: Progressing
  currentPodHash: 6756d4c4c7
  observedGeneration: 5745f7fc9c
  readyReplicas: 5
  replicas: 5
  selector: 'app=partner-support-portal,rollouts-pod-template-hash=5f59c4c4dc'

Simplified replicaset

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  annotations:
    rollout.argoproj.io/desired-replicas: '3'
    rollout.argoproj.io/revision: '16'
    rollout.argoproj.io/revision-history: '14'
  generation: 1
  labels:
    app: partner-support-portal
  name: partner-support-portal-7c67b5fd8d
  ownerReferences:
    - apiVersion: argoproj.io/v1alpha1
      blockOwnerDeletion: true
      controller: true
      kind: Rollout
      name: partner-support-portal
spec:
  minReadySeconds: 30
  replicas: 3
  selector:
    matchLabels:
      app: partner-support-portal
      rollouts-pod-template-hash: 7c67b5fd8d
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: partner-support-portal
        rollouts-pod-template-hash: 7c67b5fd8d
    spec:
      containers:
        - image: somerepo:sometag
          name: partner-support-portal
      restartPolicy: Always
status:
  availableReplicas: 3
  fullyLabeledReplicas: 3
  observedGeneration: 1
  readyReplicas: 3
  replicas: 3

from argo-rollouts.

jessesuen avatar jessesuen commented on August 28, 2024

It may have something to do with the change in replicas (scaling down from 3 to 2). We are hitting the following:

func (c *Controller) isScalingEvent(rollout *v1alpha1.Rollout, rsList []*appsv1.ReplicaSet) (bool, error) {
	if rollout.Spec.Strategy.CanaryStrategy != nil {
		return false, nil
	}
	newRS, previousRSs, err := c.getAllReplicaSetsAndSyncRevision(rollout, rsList, false)
	if err != nil {
		return false, err
	}

	allRSs := append(previousRSs, newRS)

	for _, rs := range controller.FilterActiveReplicaSets(allRSs) {
		desired, ok := annotations.GetDesiredReplicasAnnotation(rs)
		if !ok {
			continue
		}
		if desired != defaults.GetRolloutReplicasOrDefault(rollout) {
			return true, nil               // we are hitting this logic
		}
	}
	return false, nil
}

from argo-rollouts.

jessesuen avatar jessesuen commented on August 28, 2024

Tracing through the code, it appears we don't have code to handle the scaling of the preview service during a scaling event. Have a potential fix.

from argo-rollouts.

jessesuen avatar jessesuen commented on August 28, 2024

The bug is that we detect a scaling event on a preview replicaset, and so we enter the scaleBlueGreen() block of code, but the current logic does nothing to correct the replicas of the preview replicaset, thereby getting stuck in a perpetual loop and never getting to the actual rollout logic.

from argo-rollouts.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.