Comments (4)
Simplified rollout object:
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
annotations:
rollout.argoproj.io/revision: '16'
generation: 1
labels:
app: partner-support-portal
name: partner-support-portal
spec:
minReadySeconds: 30
paused: false
replicas: 2
selector:
matchLabels:
app: partner-support-portal
strategy:
blueGreen:
activeService: partner-support-portal-active
previewService: partner-support-portal-preview
template:
metadata:
labels:
app: partner-support-portal
spec:
containers:
- image: someimage:sometag
name: partner-support-portal
status:
HPAReplicas: 2
availableReplicas: 5
blueGreen:
activeSelector: 5f59c4c4dc
previewSelector: 7c67b5fd8d
canary: {}
conditions:
- lastTransitionTime: '2019-05-20T07:22:41Z'
lastUpdateTime: '2019-05-20T07:22:41Z'
message: Rollout has minimum availability
reason: AvailableReason
status: 'True'
type: Available
- lastTransitionTime: '2019-05-20T11:02:00Z'
lastUpdateTime: '2019-05-20T11:02:00Z'
message: Rollout "partner-support-portal" has timed out progressing.
reason: ProgressDeadlineExceeded
status: 'False'
type: Progressing
currentPodHash: 6756d4c4c7
observedGeneration: 5745f7fc9c
readyReplicas: 5
replicas: 5
selector: 'app=partner-support-portal,rollouts-pod-template-hash=5f59c4c4dc'
Simplified replicaset
apiVersion: apps/v1
kind: ReplicaSet
metadata:
annotations:
rollout.argoproj.io/desired-replicas: '3'
rollout.argoproj.io/revision: '16'
rollout.argoproj.io/revision-history: '14'
generation: 1
labels:
app: partner-support-portal
name: partner-support-portal-7c67b5fd8d
ownerReferences:
- apiVersion: argoproj.io/v1alpha1
blockOwnerDeletion: true
controller: true
kind: Rollout
name: partner-support-portal
spec:
minReadySeconds: 30
replicas: 3
selector:
matchLabels:
app: partner-support-portal
rollouts-pod-template-hash: 7c67b5fd8d
template:
metadata:
creationTimestamp: null
labels:
app: partner-support-portal
rollouts-pod-template-hash: 7c67b5fd8d
spec:
containers:
- image: somerepo:sometag
name: partner-support-portal
restartPolicy: Always
status:
availableReplicas: 3
fullyLabeledReplicas: 3
observedGeneration: 1
readyReplicas: 3
replicas: 3
from argo-rollouts.
It may have something to do with the change in replicas (scaling down from 3 to 2). We are hitting the following:
func (c *Controller) isScalingEvent(rollout *v1alpha1.Rollout, rsList []*appsv1.ReplicaSet) (bool, error) {
if rollout.Spec.Strategy.CanaryStrategy != nil {
return false, nil
}
newRS, previousRSs, err := c.getAllReplicaSetsAndSyncRevision(rollout, rsList, false)
if err != nil {
return false, err
}
allRSs := append(previousRSs, newRS)
for _, rs := range controller.FilterActiveReplicaSets(allRSs) {
desired, ok := annotations.GetDesiredReplicasAnnotation(rs)
if !ok {
continue
}
if desired != defaults.GetRolloutReplicasOrDefault(rollout) {
return true, nil // we are hitting this logic
}
}
return false, nil
}
from argo-rollouts.
Tracing through the code, it appears we don't have code to handle the scaling of the preview service during a scaling event. Have a potential fix.
from argo-rollouts.
The bug is that we detect a scaling event on a preview replicaset, and so we enter the scaleBlueGreen()
block of code, but the current logic does nothing to correct the replicas of the preview replicaset, thereby getting stuck in a perpetual loop and never getting to the actual rollout logic.
from argo-rollouts.
Related Issues (20)
- Split the interactive object dependency for Containers and Revision Widgets
- Argo Rollouts misinterpreting init-style sidecar containers as normal initContainers
- Allow disabling Rollbacks HOT 1
- Add Notification template in case of manual Rollback HOT 1
- In Blue-Green, Pre-Promotion jobs are getting terminated prematurely whenever application pod crashes and the whole rollout is marked successfull
- Better UI support for experiments
- Add timeout attribute for NewRelic metrics provider
- Aborting paused rollouts is not working (endless reconsiliation loop) HOT 1
- feat: enable Profiling support to detect cpu/mem bottlenecks HOT 1
- Aborting canary rollouts with ALB based traffic-routing causes toggling rollouts-pod-template-hash in preview service HOT 4
- Argo Rollouts doesn't support gracefully draining connections?
- Old replica sets not always scaling down after update from 1.6.6 to 1.7.1 HOT 2
- Canary support with argo rolllouts with kong traffic provider does not work seamlessly HOT 1
- Rollout can sometimes run indefinitely when new pods are in a crash loop HOT 2
- Synchronization barrier for deploying to multiple clusters in parallel HOT 3
- New Relic canary verification HOT 4
- Failed to compare desired state to live state
- setHeaderRoute causes sync loop when rollout steps have finished
- AbortScaleDownDelaySeconds (scale-down-deadline) not working in case of retry HOT 1
- Race condition between Argo and AWS LB Controller causing rollouts to get stuck
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from argo-rollouts.