thesandlord / container-orchestration-comparisons Goto Github PK

Hi @andrewwatson
how do we check deployment health check before Flip the traffic to "Green"
Deploy "Version 1" of an application as "Blue"
Ensure "Blue" is serving traffic
Deploy "Version 2" of an application as "Green"

now before Flip the traffic to "Green", there might be the case the deployment health check is failing due to some issue in application .
let say i have following health check defined and due to wrong commit, application is failing to launch.
livenessProbe:{
httpGet:{
path:'/buildInfo',
port: 8080
},
initialDelaySeconds:45,
timeoutSeconds:10
}

So i want to check if health check is passing for new version before Flip the traffic to "Green, other wise service will start pointing to failed version and will have downtime.

the main issue there is no way to get health check as soon as we do deployment. i tried to fetch pods status after the deployment, kubernetes is throwing error only after quite log time and during that time the result is aways showing "pods are running".
the only workaround is i need to put some wait time in our workflow and make sure all pods with new version running fine before Flip the traffic to "Green".

Do you have any other suggestion here?

I was going to ask why do you do it in this way, but I've just seen that you're following the CF doc.

It's better to update the labels in the deployment than the labels in the service. By doing it your way, there might be some downtime (when the selector changes and the endpoints are updated).

1.- Deploy version A, labels as color=Blue,access=public
2.- Create a service PUBLIC with selector access=public
3.- Deploy version B, label as color=Green
4.- Create a service INTERNAL with selector color=Green

This is the setup until Step 2 in the CF page. Now we have to update the access, this is, version B has to be accessed via the PUBLIC service. To do so, you have to add the access label to version B

 kubectl label pods -l color=green access=public

Now you have Step 3, public traffic is sent to both pods: green and blue. Then we remove the blue pod (version A)

     kubectl label pods -l color=blue access-

etc... Using labels at the right level will guarantee no issues 😄