Comments (3)
Hey Constança, thanks for opening this issue. Some comments below.
Create a new buildkite pipeline in this directory.
I don't think there is any need to create a Buildkite pipeline, since ESF does not need to be released as part of the Elastic stack. So feel free to keep using Github Actions as we already do, unless you find some benefit in moving to Buildkite.
Each version release (or commit?) triggers the creation of a new zip file with all the dependencies. This zip file needs to be pushed to an S3 bucket that will be used by customers.
We currently track releases with git tags, so the workflow could be triggered by the creation of a new git tag. We also track the version in version.py, which is currently updated manually. There is already a related issue about how to automate updates on this file and how to handle version bumps in general #540. I'd consider it as a preliminary task for this issue.
I would also make sure that the solution is extensible enough to be able to add automated deployment to SAR as well, in a future release.
Remove the module for lambda.
Insert a new resource aws_lambda_function that reads from the S3 bucket with the zip file:
This is more like an option, the current AWS Lambda Terraform module terraform-aws-modules/lambda/aws
can still be used if it simplifies the implementation. It just need to be modified to use pre-built packages stored on S3
Where should the S3 bucket be placed? Under which account? In any specific region?
You can use the same account where we store SAR artifacts.
Do all packages used in import statements in the handlers files need to be in the dependency zip?
Technically no, the AWS Lambda Python runtime already includes some of them (e.g boto3). However, we should stick to what is on requirements.txt
to be sure to use the same versions everywhere. You should include only the dependencies used at runtime (i.e only requirements.txt). This part also depends on whether #204 is going to be prioritized or not.
from elastic-serverless-forwarder.
Hey @girodav and @axw , can I have your thoughts on this to make sure everything is correct?
from elastic-serverless-forwarder.
Thank you @girodav for such a detailed answer. I am working on setting a workflow on github actions like you mentioned. It seems a bit tricky to test, so I will do it in a private repository first and then I will open a PR and link it to this issue as well as to #540.
It won't be taking of the SAR currently but it seems easy to adapt the workflow if setting the right trigger:
on:
push:
branches:
- 'main'
paths:
- 'version.py'
from elastic-serverless-forwarder.
Related Issues (20)
- permission for elastic-serverless-forwarder deployment user
- v1.11 missing outputs Lambda function ARN and IAM Role ARN HOT 2
- Invalid request provided: AWS::Logs::SubscriptionFilter HOT 4
- [DOCS] Document JSON content format we support
- [DOCS] Add missing documentation for `root_fields_to_add_to_expanded_event`
- Terraform definition HOT 1
- Test and support arm64
- Incorrect Handling of Permissions Can Result in Data Loss
- Provide support for Python 3.12
- [DOCS] Fix outdated link
- CloudWatch log retention for ESF own logs
- Investigate ingestion of files with same IDs HOT 2
- Misconfigured ESF input results in a warning message, not events sent to replay queue HOT 3
- CMA-CGM ESF Issue HOT 1
- no input defined on cloudwatch-logs trigger HOT 1
- Enable shared outputs and multiple outputs per input
- Investigate number of API calls of ESF due to ec2:describe_regions usage HOT 3
- Issue with region name check in publishing script
- Support SSM (Systems Manager) parameters
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from elastic-serverless-forwarder.