manheim / cf_deployer Goto Github PK

We use log4r internally in the gem, it would be better if we defaulted to something internal to ruby (removing dependency) and allowed users to configure the gem to use something like log4r if that's what they wanted.

• Assume the current deployed stack is blue
• The blue stack has asgs ASG1 and ASG2
• ASG3 is added to the CF template
• The application is deployed

When spinning up the green stack, it looks for ASG3 in the blue stack to determine the desired number of instances. This is not a CF stack output, which raises an error.
Regression from #9

I ran across some unexpected behavior while trying to update a stack's tags. I updated my configuration, ran the deploy and received back the message No updates are to be performed.

I had expected cf_deployer to send the new tags to the stack and perform an update.

The update_stack method on the Stack object does does not merge tags in to the call arguments.

The create_stack method on the Stack object does merge in the tags.

Looking at the SDK docs it appears that the API call supports updating tags.

Is there a reason the tool does not support updating tags?

Rather than throwing a stack trace, it should capture the error and just say that it can't do a diff because there's no current stack to diff against.

We're still using AWS-SDK v1. Consider what it will take to upgrade to AWS-SDK v2.

If the min/desired/max of a Blue stack have been modified manually, when Green is deployed, it only warms the new stack to the settings in the JSON instead of to what's set on the Blue stack. It should respect the Blue stack's settings and throw a warning.

If you declare a depends-on with a typo in the name, you get a weird error when trying to get the outputs of nil instead of an error or warning that the component that is declared as a dependency is undefined.

If an environment specifies an input override that is itself a hash, the current behavior attempts to resolve that from the component referenced by the value of the ':component' nested param. E.g.

some-env:
  components:
    some-component:
      inputs:
        some-key:
          component: parent
          output-key: parent-key

However, this disallows overriding any nested hash. This should only attempt to resolve from the referenced component when another component is actually referenced.

In calls like healthy_instance_count, we're making N+1 calls to AWS to pull the individual health of each instance. Use the memoize method to reduce the number of calls to AWS.

There will be a separate issue to upgrade to AWS v2, which handles response caching differently. (See Issue #49)

We've seen issues in AWS, where a stack can get caught in UPDATE_IN_PROGRESS permanently. During a deployment, we poll the status for the stack, and wait until it either has a success, or a failure status - if the stack is perpetually in neither of those states, the polling (and therefore the deployment) will continue forever.

Add a default/configurable timeout around checking the stack status, so that such problems can be detected in an automated pipeline.

https://github.com/manheim/cf_deployer/blob/master/lib/cf_deployer/stack.rb#L156

Noted by @ryanrwalker during code review for Issue #38.

During a deployment, if a stack update fails, the change is automatically rolled back to the previous state. When this happens using cf_deployer, cf_deployer returns a success (zero) status, suggesting that there was no problem. If a rollback results from a failing deployment, cf_deployer should return a non-success (non-zero) status.

If you enable debugging, you can see that we make a large number of duplicate requests to the AWS-SDK. Calls to the AWS-SDK can fail, and the gem provides an automated exponential retry for failed requests. If the maximum number of retries is exceeded, cf_deployer can fail in the middle of a deployment.

There are a couple of options here:
- Make retries configurable
- reduce the number of aws-sdk calls, through refactoring, or the use of AWS.memoize
- replace aws-sdk-v1 with aws-sdk-v2

No test was ever written for when switch is called and no stack exists. When switch is called in this case, it displays the error mesasge 'Only one color stack exists...' which is not actually the case.

Update the error message to take this case into account.

This method sleeps until the instance count is exactly equal to the desired. If auto-scaling were to scale the group higher, this would continue to sleep. It should be changed to sleep while < desired.

Note: We probably haven't run into this yet because it happens during the intermediate stage when both stacks are up, which would generally appear to be low load to either group.

Using the ASG-based blue/green deployment strategy, if the deployment fails (particularly during warm-up), both the blue and green ASGs can end up with instances, effectively leaving both stacks "active". It requires manual intervention to get the application deployable (cf_deployer will raise an error if both ASGs are active), and in the interim, customer requests can be routed to the possibly incompletely/incorrectly deployed ASG.

Ensure that a failure during the ASG-based blue/green deployment automatically "rolls back" - ie: sets the min/max of the new ASG to 0.