A Go based CLI for removing unused versions of AWS Lambdas. One binary, no additional dependencies required.
go-lambda-cleanup is distributed as a single binary. Download the binary and install go-lambda-cleanup in a directory in your system's PATH. /usr/local/bin
is the recommended path for UNIX/LINUX environments.
VERSION=2.0.2
wget https://github.com/karl-cardenas-coding/go-lambda-cleanup/releases/download/v$VERSION/go-lambda-cleanup-v$VERSION-linux-amd64.zip
unzip go-lambda-cleanup-v$VERSION-linux-amd64.zip
sudo mv glc /usr/local/bin/
go-lambda-cleanup is also available as a Docker image. Check out the GitHub Packages page for this repository to learn more about the available images.
VERSION=2.0.2
docker pull ghcr.io/karl-cardenas-coding/go-lambda-cleanup:$VERSION
You can pass AWS credentials to the container through ENVIRONMENT variables.
export AWS_ACCESS_KEY_ID=47as12fdsdg....
export AWS_SECRET_ACCESS_KEY=21a5sf5dg8e...
docker run -e AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY ghcr.io/karl-cardenas-coding/go-lambda-cleanup:$VERSION clean -r us-east-1 -d
time=05/23/22 level=info msg="******** DRY RUN MODE ENABLED ********"
time=05/23/22 level=info msg="Scanning AWS environment in us-east-1"
time=05/23/22 level=info msg=............
time=05/23/22 level=info msg="8 Lambdas identified"
time=05/23/22 level=info msg="Current storage size: 193 MB"
time=05/23/22 level=info msg="**************************"
time=05/23/22 level=info msg="Initiating clean-up process. This may take a few minutes...."
time=05/23/22 level=info msg=............
time=05/23/22 level=info msg=............
time=05/23/22 level=info msg="24 unique versions will be removed in an actual execution."
time=05/23/22 level=info msg="124 MB of storage space will be removed in an actual execution."
time=05/23/22 level=info msg="Job Duration Time: 1.454406s"
Usage:
glc [flags]
glc [command]
Available Commands:
clean Removes all former versions of AWS lambdas except for the $LATEST version
help Help about any command
version Print the current version number of glc
Flags:
-d, --dryrun Executes a dry run (bool)
-h, --help help for glc
-l, --listFile string Specify a file containing Lambdas to delete.
-m, --moreLambdaDetails Set to true to show Lambda names and count of versions to be removed (bool)
-p, --profile string Specify the AWS profile to leverage for authentication.
-r, --region string Specify the desired AWS region to target.
-i, --size-iec Displays file sizes in IEC units (bool)
-v, --verbose Set to true to enable debugging (bool)
Use "glc [command] --help" for more information about a command.
To retain previous version excluding $LATEST
, use the -c
flag. Use this flag to control the number of versions to retain.
$ glc clean -r us-east-2 -c 2 -p myProfile
To view additional details, such as the Lambda names and version counts, set the -m
flag to true.
glc clean -r us-east-1 -dmp mySuperAwesomeProfile
INFO[07/31/22] Scanning AWS environment in us-east-1
INFO[07/31/22] ............
INFO[07/31/22] 72 Lambdas identified
INFO[07/31/22] Current storage size: 817 MiB
INFO[07/31/22] **************************
INFO[07/31/22] Initiating clean-up process. This may take a few minutes....
INFO[07/31/22] ............
INFO[07/31/22] 1 versions of YourLambda to be removed
INFO[07/31/22] 1 versions of AnotherLambda to be removed
INFO[07/31/22] ............
INFO[07/31/22] 2 unique versions will be removed in an actual execution.
INFO[07/31/22] 12 MiB of storage space will be removed in an actual execution.
INFO[07/31/22] Job Duration Time: 9.409405s
To display file sizes in IEC format, enable the -i
flag.
$ glc clean -i -r us-east-1 -p myProfile
You also have the ability to preview an execution by leveraging the dry run flag -d
$ glc clean -p myProfile -r us-east-1 -d
INFO[03/19/21] The AWS Profile flag "myProfile" was passed in
INFO[03/19/21] ******** DRY RUN MODE ENABLED ********
INFO[03/19/21] Scanning AWS environment in us-east-1
INFO[03/19/21] ............
INFO[03/19/21] 50 Lambdas identified
INFO[03/19/21] Current storage size: 1.2 GB
INFO[03/19/21] **************************
INFO[03/19/21] Initiating clean-up process. This may take a few minutes....
INFO[03/19/21] ............
INFO[03/19/21] ............
INFO[03/19/21] 82 unique versions will be removed in an actual execution.
INFO[03/19/21] 554 MB of storage space will be removed in an actual execution.
INFO[03/19/21] Job Duration Time: 7.834585s
You can provide an input file containing a list of Lambda functions to be cleaned-up. The input file can be of the following types; json
, yaml
, or yml.
An input file allows you to control the execution more granularly.
# custom_list.yaml
lambdas:
- stopEC2-instances
- putControls
$ glc clean -r us-east-1 -p myProfile -l custom_list.yaml
{
"lambdas": [
"stopEC2-instances",
"putControls"
]
}
glc clean -r us-east-1 -p myProfile -l custom_list.json
go-lambda-clean utilizes the default AWS Go SDK credentials provider to find AWS credentials. The default provider chain looks for credentials in the following order:
-
Environment variables.
-
Shared credentials file.
-
If your application uses an ECS task definition or RunTask API operation, IAM role for tasks.
-
If your application is running on an Amazon EC2 instance, IAM role for Amazon EC2.
If ~/.aws/config
and ~/.aws/config
is setup for the AWS CLI then you may leverage the existing profile configuration for authentication.
$ export AWS_PROFILE=sb-test
$ glc clean -r us-west-2
INFO[03/05/21] Scanning AWS environment in us-west-2
INFO[03/05/21] ............
Alternatively, the --profile
flag may be used.
$ glc clean -r us-west-2 -p myProfile
INFO[03/05/21] Scanning AWS environment in us-west-2
INFO[03/05/21] ............
Static credentials may be also be used to authenticate into AWS.
-
AWS_ACCESS_KEY_ID
-
AWS_SECRET_ACCESS_KEY
-
AWS_SESSION_TOKEN (optional)
$ export AWS_ACCESS_KEY_ID=YOUR_AKID
$ export AWS_SECRET_ACCESS_KEY=YOUR_SECRET_KEY
$ export AWS_SESSION_TOKEN=TOKEN
$ glc clean -r us-west-2
2021/03/04 20:42:46 Scanning AWS environment in us-west-2.....
2021/03/04 20:42:46 ............
If you want to complile the binary, clone the project to your local system. Ensure you have Go 1.18
installed. This tool leverages the Golang embed functionality. A file named aws-regions.txt
is expected in the cmd/
directory. You need valid AWS credentials in order to generate the file.
git clone [email protected]:karl-cardenas-coding/go-lambda-cleanup.git
aws ec2 describe-regions --region us-east-1 --all-regions --query "Regions[].RegionName" --output text >> cmd/aws-regions.txt
go build -o glc
The tool supports network proxy configurations and will honor the following proxy environment variables.
HTTP_PROXY
,HTTPS_PROXY
NO_PROXY
The environment values may be either a complete URL or a "host[:port]", in which case the "http" scheme is assumed. An error is returned if the value is a different form.
$ export HTTP_PROXY=http://proxy.example.org:9000
$ glc clean -r us-west-2
2021/03/04 20:42:46 Scanning AWS environment in us-west-2.....
2021/03/04 20:42:46 ............
go-lambda-cleanup is a good fit for cron jobs. Below is an example snippet for how you can setup a cron job through GitHub Actions.
name: Nightly Lambda Version Cleanup
on:
schedule:
# At 04:00 on every day
- cron: '0 04 * * *'
env:
VERSION: 2.0.0
jobs:
build:
name: Run go-lambda-cleanup
runs-on: ubuntu-latest
steps:
- name: Pull Docker Image
run: docker pull ghcr.io/karl-cardenas-coding/go-lambda-cleanup:$VERSION
- name: Run go-lambda-cleanup in Test
env:
AWS_ACCESS_KEY_ID: ${{secrets.AWS_TEST_ACCESS_KEY}}
AWS_SECRET_ACCESS_KEY: ${{secrets.AWS_TEST_SECRET_ACCESS_KEY}}
REGION: us-east-1
run: docker run -e AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY ghcr.io/karl-cardenas-coding/go-lambda-cleanup:$VERSION clean -r $REGION
- name: Run go-lambda-cleanup in Prod
env:
AWS_ACCESS_KEY_ID: ${{secrets.AWS_PROD_ACCESS_KEY}}
AWS_SECRET_ACCESS_KEY: ${{secrets.AWS_PROD_SECRET_ACCESS_KEY}}
REGION: us-east-1
run: docker run -e AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY ghcr.io/karl-cardenas-coding/go-lambda-cleanup:$VERSION clean -r $REGION
For a complete guide to contributing to go-lambda-clean, please review the Contribution Guide.
Contributions to go-lambda-cleanup of any kind are welcome. Contributions include, but not limited to; documentation, organization, tutorials, blog posts, bug reports, issues, feature requests, feature implementations, pull requests, answering questions on the forum, helping to manage issues, etc.
Q: On MacOS I am unable to open the binary due to Apple not trusting the binary. What are my options?
A: You have four options.
-
Option A (Recommended) is to use the Docker container. Please review the Docker steps.
-
Option B is to to grant permission for the application to run. Use this guide to help you grant permission to the application.
-
Option C is not recommended but I'll offer it up. You can remove the binary from quarantine mode.
xattr -d com.apple.quarantine /path/to/file
- Option D is to clone this project and compile the binary. Issue
go build -o glc
, and the end result is a binary compatible for your system. If you still encounter issues after this, invoke the code signing command on the binarycodesign -s -
Q: This keeps timing out when attempting to connect to AWS and I have verified my AWS credentials are valid?
A: This could be related to a corporate firewall. If your organization has a proxy endpoint configure the proxy environment variable with the correct proxy endpoint. Consult your organization's networking team to learn more about the proper proxy settings.
Q: I don't want to execute this command without understanding exactly what it will do. Is there a way to preview the actions?
A: Yes, leverage the dry run mode. Dry run can be invoked through the -d
, --dryrun
flag.