This template demonstrates how to develop and deploy a simple SQS-based producer-consumer service running on AWS Lambda using the Serverless Framework and the Lift plugin. It allows to accept messages, for which computation might be time or resource intensive, and offload their processing to an asynchronous background process for a faster and more resilient system.

This template defines one function producer and one Lift construct - jobs. The producer function is triggered by http event type, accepts JSON payloads and sends it to a SQS queue for asynchronous processing. The SQS queue is created by the jobs queue construct of the Lift plugin. The queue is set up with a "dead-letter queue" (to receive failed messages) and a worker Lambda function that processes the SQS messages.

To learn more:

• about http event configuration options, refer to http event docs
• about the queue construct, refer to the queue documentation in Lift
• about the Lift plugin in general, refer to the Lift project
• about SQS processing with AWS Lambda, please refer to the official AWS documentation

Install dependencies with:

npm install

Then deploy:

serverless deploy

After running deploy, you should see output similar to:

Deploying aws-python-sqs-worker-project to stage dev (us-east-1)

✔ Service deployed to stack aws-python-sqs-worker-project-dev (175s)

endpoint: POST - https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/produce
functions:
  producer: aws-python-sqs-worker-project-dev-producer (167 kB)
  jobsWorker: aws-python-sqs-worker-project-dev-jobsWorker (167 kB)
jobs: https://sqs.us-east-1.amazonaws.com/000000000000/aws-python-sqs-worker-project-dev-jobs

Note: In current form, after deployment, your API is public and can be invoked by anyone. For production deployments, you might want to configure an authorizer. For details on how to do that, refer to http event docs.

After successful deployment, you can now call the created API endpoint with POST request to invoke producer function:

curl --request POST 'https://xxxxxx.execute-api.us-east-1.amazonaws.com/produce' --header 'Content-Type: application/json' --data-raw '{"name": "John"}'

In response, you should see output similar to:

{"message": "Message accepted!"}

In case you would like to include 3rd party dependencies, you will need to use a plugin called serverless-python-requirements. You can set it up by running the following command:

serverless plugin install -n serverless-python-requirements

Running the above will automatically add serverless-python-requirements to plugins section in your serverless.yml file and add it as a devDependency to package.json file. The package.json file will be automatically created if it doesn't exist beforehand. Now you will be able to add your dependencies to requirements.txt file (Pipfile and pyproject.toml is also supported but requires additional configuration) and they will be automatically injected to Lambda package during build process. For more details about the plugin's configuration, please refer to official documentation.