An AWS Lambda Function that Sends Email via Simple Email Service (SES) and handles notifications for bounces, etc. 📈
We send (and receive) a lot of email
both for the @dwyl App
and our newsletter.
We need a simple, scalable & maintainable way of sending email,
and most importantly we needed to know with certainty:
- Are our emails being delivered?
- How many emails are bouncing?
- Are we attempting to re-send email to addresses that have "bounced"? (i.e. wasting money?!)
- Are people opening / reading the email?
- Do people engage with the content of the email? (click through)
- If someone no longer wants to receive emails (too many or not relevant), do we have a reliable way for them to unsubscribe?
This project is our quest to answer these questions.
The aws-ses-lambda function does three related things1:
- Send emails.
- Parse AWS SNS notifications related to the emails that were sent.
- Save the parsed SNS notification data for aggregation and visualisation.
The How? section below explains how each of these functions works.
This diagram explains the context where aws-ses-lambda is used:
The Email App receives requests from the Auth App
and and triggers the aws-ses-lambda function.
The aws-ses-lambda function Sends email
and handles SNS notifications for bounce events.
As the name of this project suggests, we are using AWS Lambda, to handle all email-related tasks via AWS SES.
If you (or anyone
elseon your team) are new to AWS Lambda, see: github.com/dwyl/learn-aws-lambda
In this section we will break down how the lambda works.
Thanks to the work we did earlier on
sendemail,
sending emails using AWS Simple Email Service (SES)
from our Lambda function is very simple.
We just need to follow the setup instructions in
github.com/dwyl/sendemail#how
including creating a /templates directory,
then create a handler function:
const sendemail = require('sendemail').email;
module.exports = function send (event, callback) {
return sendemail(event.template, event, callback);
};Don't you just love it when things are that simple?!
All the data required for sending an email
is received in the Lambda event object.
The required keys in the event object are:
email- the email address we want to send an email to.name- the name of the person we are sending the email to. (if your email messages aren't personal, don't send them!)subject- the subject of the email you are sending.template- the template you want to send.
It works flawlessly.
The full code is:
lib/send.js
After an email is sent using AWS SES,
AWS keeps track of the status of the emails
e.g delivered, bounce or complaint.
By subscribing to AWS Simple Notification System (SNS)
notifications, we can keep track of the status.
There are a few steps
for setting up SNS notifications for SES events,
so we created detailed setup instructions:
SETUP.md
Once you have configured the SNS Topic, used the topic for SES notifications and set the topic as the trigger for the lambda function, it's time to parse the notifications.
Thankfully this is also really simple code!
let json = {};
if(event && event.Records && event.Records.length > 0) {
const msg = JSON.parse(event.Records[0].Sns.Message);
json.messageId = msg.mail.messageId;
json.notificationType = msg.notificationType + ' ' + msg.bounce.bounceType;
}We are only interested in the messageId and notificationType.
This code is included in
lib/parse.js
During MVP we are only interested in the emails that bounce. So we are only parsing the bounce event. Gmail does not send delivery notifications, so we will need to implement a workaround. See: dwyl/email#1
More detail on the various SES SNS notifications: https://docs.aws.amazon.com/ses/latest/DeveloperGuide/event-publishing-retrieving-sns-examples.html
Once we have parsed the SNS notifications for SES events, we need to save the data back to our PostgreSQL database so that we can build our analytics dashboard!
This again is pretty simple code;
we just invoke http_request
with the json data we want to send to the Phoenix App:
const json = parse(event); // parse SNS event see: step 2.
http_request(json, callback); // json data & lambda callback argumentView the complete code in index.js
and the supporting http_request function in
lib/http_request.js
The http_request function wraps the Node.js core
http.request
method with a few basic options
and allows us to pass in a json Object
to send to the Phoenix App.
In order for all parts of the Lambda function to work, we need to ensure that all environment variables are defined.
For the complete list of required environment variables,
please see the .env_sample file.
Copy the .env_sample file and create a .env file:
cp .env_sample .env
Then update all the values in the file so that they are the real values.
Once you have a .env file with all the correct environment variables,
it's time to deploy the Lambda function to AWS!
Run the following command in your terminal:
npm run deploy
You should see output similar to the following:
- - - - - - - - > Lambda Function Deployed:
{
FunctionName: 'aws-ses-lambda-v1',
FunctionArn: 'arn:aws:lambda:eu-west-1:123456789247:function:aws-ses-lambda-v1',
Runtime: 'nodejs12.x',
Role: 'arn:aws:iam::123456789247:role/service-role/LambdaExecRole',
Handler: 'index.handler',
CodeSize: 8091768,
Description: 'A complete solution for sending email via AWS SES using Lambda',
Timeout: 42,
MemorySize: 128,
LastModified: '2020-03-05T23:42:56.809+0000',
CodeSha256: 'jvOg/+8y9UwBcLeTprMRIEvT0ryun1bdjzrAJXAk5m8=',
Version: '$LATEST',
Environment: { Variables: { EMAIL_APP_URL: 'phemail.herokuapp.com' } },
TracingConfig: { Mode: 'PassThrough' },
RevisionId: '42442cee-d506-4aa5-aec5-d7fb73145a58',
State: 'Active',
LastUpdateStatus: 'Successful'
}
- - - - - - - - > took 8.767 secondsEnsure you follow all the instructions in
SETUP.md
to get the SNS Topic to trigger the Lambda function for SES notifications.
Enable debugging by setting the NODE_ENV=test environment variable.
Now the latest event will be saved to:
https://ademoapp.s3.eu-west-1.amazonaws.com/event.json
And SNS messages are saved to: https://ademoapp.s3.eu-west-1.amazonaws.com/sns.json
There are way more reasons
why we are handcrafting this app
than the ones stated above.
We see email as our primary feedback mechanism
and thus "operationally strategic",
not merely "transactional".
i.e. not something to be "outsourced"
to a "black box" provider that "takes care of everything" for us.
We want to have full control and deep insights into our email system.
By using a decoupled lambda function to send email
and subscribe to SNS events
we keep all the AWS specific
functionality in a single place.
This is easy to reason about, maintain and extend when required.
In the future, if we decide to switch email sending provider,
(or run our own email service),
we can simply re-write the sendemail
and parse_notification functions
and not need to touch our email analytics dashboard at all!
For now SES is by far the cheapest and superbly reliable way to send email. We are very happy to let AWS take care of this part of our stack.
1 The aws-ses-lambda function does 3 things
because they relate to the unifying theme of
sending email via SES and tracking the status of the sent emails via SNS.
We could have split these 3 bits of functionality into separate repositories
and deploy them separately as distinct lambda functions,
however in our experience having too many lambda functions
can quickly become a maintenance headache.
We chose to group them together
because they are small, easy to reason about
and work well as a team!
If you feel strongly about the
UNIX Philosophy
definitely split out the functions in your own fork/implementation.
The code for this Lambda function is less than
100 lines
and can be read in 10 minutes.
The sendemail module
which the Lambda uses to send emails via AWS SES is 38 lines of code. See:
lib/index.js
it's mostly comments which make it very beginner friendly.
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent