Code Monkey home page Code Monkey logo

sar-lambda-janitor's Introduction

SAR-Lambda-Janitor

Version Greenkeeper badge CircleCI License

Cron job for deleting old, unused versions of your Function.

This post explains the problem and why we created this app.

Safeguards

To guard against deleting live versions, some safeguards are in place:

  • Never delete the $LATEST version. This is the default version that will be used when you invoke a function.
  • Never delete versions that are referenced by an alias. If you use aliases to manage different stages - dev, staging, etc. then the latest version referenced by your aliases will not be deleted.
  • Keeping the most recent N versions. Even if you don't use aliases at all, we will always keep the most recent N versions, where N can be configured with the VersionsToKeep parameter when you install the app. Defaults to 3.

Deploying to your account (via the console)

Go to this page and click the Deploy button.

This app would deploy the following resources to your region:

  • a Lambda function that scans the functions in your region and deletes unused versions
  • a CloudWatch event schedule that triggers the Lambda function every hour

Deploying via SAM/Serverless framework/CloudFormation

To deploy this app via SAM, you need something like this in the CloudFormation template:

AutoDeployMyAwesomeLambdaLayer:
  Type: AWS::Serverless::Application
  Properties:
    Location:
      ApplicationId: arn:aws:serverlessrepo:us-east-1:374852340823:applications/lambda-janitor
      SemanticVersion: <enter latest version>
    Parameters:
      VersionsToKeep: <defaults to 3>

To do the same via CloudFormation or the Serverless framework, you need to first add the following Transform:

Transform: AWS::Serverless-2016-10-31

For more details, read this post.

sar-lambda-janitor's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sar-lambda-janitor's Issues

An in-range update of husky is breaking the build 🚨

The devDependency husky was updated from 3.0.9 to 3.1.0.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

husky is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details
  • ci/circleci: build: Your tests failed on CircleCI (Details).

Release Notes for v3.1.0
  • Add pre-merge-commit hook (#605)
Commits

The new version differs by 5 commits.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

Action required: Greenkeeper could not be activated 🚨

🚨 You need to enable Continuous Integration on Greenkeeper branches of this repository. 🚨

To enable Greenkeeper, you need to make sure that a commit status is reported on all branches. This is required by Greenkeeper because it uses your CI build statuses to figure out when to notify you about breaking changes.

Since we didn’t receive a CI status on the greenkeeper/initial branch, it’s possible that you don’t have CI set up yet.
We recommend using:

If you have already set up a CI for this repository, you might need to check how it’s configured. Make sure it is set to run on all new branches. If you don’t want it to run on absolutely every branch, you can whitelist branches starting with greenkeeper/.

Once you have installed and configured CI on this repository correctly, you’ll need to re-trigger Greenkeeper’s initial pull request. To do this, please click the 'fix repo' button on account.greenkeeper.io.

Feature: Add No-op option

When first applying to a production environment, I would like to be able to first see what would be done before doing it.

Add safety features to prevent accidental deletion of running functions

Very cool app! I'm concerned that the initial version of this app is too aggressive in which function versions get deleted. If I'm reading the code correctly, it looks like the only function versions it won't delete are ones that have an alias pointing at them. However, not everyone uses aliases on their functions so the app could delete a function that's serving customer traffic, causing an outage. The safety feature I've seen added to similar apps is to have a configurable minimum number of versions to keep for a given function. Keeping the newest n versions of the function around usually provides a sufficient safety net.

Publish in AWS China

Hi! Are there any plans to publish this SAR in AWS China? We use it in most of our other AWS accounts and would like too use it in China too.

Don't delete lambda versions if the stack is in UPDATE_ROLLBACK_FAILED state

Hi,

If the "keep most recent N versions" is set to 1 then it's possible the janitor will clean up lambda versions even if a cloudformation stack is stuck in UPDATE_ROLLBACK_FAILED state. In this case if the janitor happens to run before continuing rollback in cloudformation then this can cause the stack to be stuck as it won't be able to revert to the previous lambda versions (as they've since been deleted).

There's various ways to get it back to a sensible state, but it's rather cumbersome with the AWS console as it often fails to do so properly.

Normally the best solution is to keep a sensible value of at least 2 for the number of versions to keep.

Unfortunately in our case we were reaching the limits of Lambda storage and had to force ourselves to keep 1 lambda version instead of more.


Would it be possible to add a condition in the janitor to keep at least 2 lambda versions if the associated stack is in a failed state, or otherwise respect the configured value of 1 if the stack is in UPDATE_ROLLBACK_COMPLETE or UPDATE_COMPLETE?

I can understand if this is a big request, we can come up with alternatives in the meantime.

Cheers!

Cleanup fails with TooManyRequestsException.

Cleanup fails with TooManyRequestsException.
{ "message": "retrying listAliases after error...", "function": "arn:aws:lambda:us-east-1:xxxxxxxxx:function:stg-xxxxx-xxxxx--xxxxxxxx-Zg5KOhlu8oeX", "errorName": "TooManyRequestsException", "errorMessage": "Rate exceeded", "stackTrace": "TooManyRequestsException: Rate exceeded\n at Object.extractError (/var/task/node_modules/aws-sdk/lib/protocol/json.js:51:27)\n at Request.extractError (/var/task/node_modules/aws-sdk/lib/protocol/rest_json.js:55:8)\n at Request.callListeners (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:106:20)\n at Request.emit (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:78:10)\n at Request.emit (/var/task/node_modules/aws-sdk/lib/request.js:683:14)\n at Request.transition (/var/task/node_modules/aws-sdk/lib/request.js:22:10)\n at AcceptorStateMachine.runTo (/var/task/node_modules/aws-sdk/lib/state_machine.js:14:12)\n at /var/task/node_modules/aws-sdk/lib/state_machine.js:26:10\n at Request.<anonymous> (/var/task/node_modules/aws-sdk/lib/request.js:38:9)\n at Request.<anonymous> (/var/task/node_modules/aws-sdk/lib/request.js:685:12)", "awsRegion": "us-east-1", "functionName": "serverlessrepo-lambda-janitor-Clean-m4tgnmdyQTPF", "functionVersion": "$LATEST", "functionMemorySize": "128", "level": 40, "sLevel": "WARN" }

"Rate Limit Exceeded" on 300+ functions

Hello!

I'm face some problems when running Janitor on environments that have 300+ functions, and getting "Rate Limit Exceeded" on lambda API. There's a way to make the scanner run syncronous or more slower on large environments?

Thanks!

[Feature Request] Custom clean up schedule (rate)

Would you be willing to accept a pull request to parameterize the schedule for invoking the Lambda function in your CloudFormation template that is currently hard coded to rate(1 hour)? (Template.yaml line 41)

The proposed change would be to add an optional parameter with the default of the current value (rate(1 hour)).

This would allow us to invoke it much less frequently for some scenarios where changes are less frequent.

An in-range update of commitlint is breaking the build 🚨

There have been updates to the commitlint monorepo:

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

This monorepo update includes releases of one or more dependencies which all belong to the commitlint group definition.

commitlint is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details
  • ci/circleci: build: Your tests failed on CircleCI (Details).

Commits

The new version differs by 39 commits.

  • c17420d v8.1.0
  • ca19d70 chore: update dependency lodash to v4.17.14 (#724)
  • 5757ef2 build(deps): bump lodash.template from 4.4.0 to 4.5.0 (#721)
  • 5b5f855 build(deps): bump lodash.merge from 4.6.0 to 4.6.2 (#722)
  • 4cb979d build(deps): bump lodash from 4.17.11 to 4.17.13 (#723)
  • a89c1ba chore: add devcontainer setup
  • 9aa5709 chore: pin dependencies (#714)
  • c9ef5e2 chore: centralize typescript and jest setups (#710)
  • c9dcf1a chore: pin dependencies (#708)
  • 6a6a8b0 refactor: rewrite top level to typescript (#679)
  • 0fedbc0 chore: update dependency @types/jest to v24.0.15 (#694)
  • 0b9c7ed chore: update dependency typescript to v3.5.2 (#695)
  • 4efb34b chore: update dependency globby to v10 (#705)
  • 804af8b chore: update dependency lint-staged to v8.2.1 (#696)
  • 9075844 fix: add explicit dependency on chalk (#687)

There are 39 commits in total.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

Add guardrail against recently invoked versions

Suggested by @jlhood in #1 here.

Need to confirm whether metrics are tracked against the version dimension when invoked through the alias ARN. When I tested this before (via a weighted alias) the count metric was only recorded for the alias but not the underlying version. It might be a non-issue though since we don't delete versions that are tagged by an alias already.

Error when trying to delete a version that was replicated to other regions

I found that lambda-janitor function has been in our us-east-1 region. After looking at CloudWatch Logs, I can see that the janitor is failing when trying to delete an older version that's still associated with a CloudFront distribution. This association to the older version is caused by a mistake on our end.

It looks like that the lambda-janitor fails after encountering this error message and fails to clean up any other versions.
Expected behavior will be to highlight this error and continue with the clean up.

Error message from CloudWatch Logs:

{
    "errorType": "InvalidParameterValueException",
    "errorMessage": "Lambda was unable to delete arn:aws:lambda:us-east-1:1234567890:function:lambda-foo:12 because it is a replicated function. Please see our documentation for Deleting Lambda@Edge Functions and Replicas.",
    "code": "InvalidParameterValueException",
    "message": "Lambda was unable to delete arn:aws:lambda:us-east-1:1234567890:function:lambda-foo:12 because it is a replicated function. Please see our documentation for Deleting Lambda@Edge Functions and Replicas.",
    "time": "2021-12-07T10:00:07.356Z",
    "requestId": "59ea2895-ffa7-4e0e-9489-5ebc7fd6edbf",
    "statusCode": 400,
    "retryable": false,
    "retryDelay": 99.4330057833587,
    "stack": [
        "InvalidParameterValueException: Lambda was unable to delete arn:aws:lambda:us-east-1:1234567890:function:lambda-foo:12 because it is a replicated function. Please see our documentation for Deleting Lambda@Edge Functions and Replicas.",
        "    at Object.extractError (/var/task/node_modules/aws-sdk/lib/protocol/json.js:52:27)",
        "    at Request.extractError (/var/task/node_modules/aws-sdk/lib/protocol/rest_json.js:55:8)",
        "    at Request.callListeners (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:106:20)",
        "    at Request.emit (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:78:10)",
        "    at Request.emit (/var/task/node_modules/aws-sdk/lib/request.js:688:14)",
        "    at Request.transition (/var/task/node_modules/aws-sdk/lib/request.js:22:10)",
        "    at AcceptorStateMachine.runTo (/var/task/node_modules/aws-sdk/lib/state_machine.js:14:12)",
        "    at /var/task/node_modules/aws-sdk/lib/state_machine.js:26:10",
        "    at Request.<anonymous> (/var/task/node_modules/aws-sdk/lib/request.js:38:9)",
        "    at Request.<anonymous> (/var/task/node_modules/aws-sdk/lib/request.js:690:12)"
    ]
}

Unable to deploy via SAM. Getting error "Plugin 'ServerlessAppPlugin' raised an exception: AWS Region cannot be found"

Hi, I'm trying to deploy SAR-Lambda-Janitor via a SAM template. I've added it to the Resources section of my SAM template:

Resources:
  SarLambdaJanitor:
    Type: AWS::Serverless::Application
    Properties:
      Location:
        ApplicationId: arn:aws:serverlessrepo:us-east-1:374852340823:applications/lambda-janitor
        SemanticVersion: 1.5.0
      Parameters:
        VersionsToKeep: 3

However, when I run:
$ sam build --debug -t ./template.yaml

I am seeing the error:
Plugin 'ServerlessAppPlugin' raised an exception: AWS Region cannot be found

I am not sure how to solve this. Any help would be much appreciated. Thank you!

An in-range update of aws-sdk is breaking the build 🚨

The dependency aws-sdk was updated from 2.569.0 to 2.570.0.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

aws-sdk is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details
  • ci/circleci: build: Your tests failed on CircleCI (Details).

Release Notes for Release v2.570.0

See changelog for more information.

Commits

The new version differs by 1 commits.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.