jeremydaly / lambda-api Goto Github PK

Now that v0.5 removes the Bluebird dependency, tests should be updated to use async/await instead of promises.

Was thinking that you could define a route for multiple methods using an array of methods in a METHOD call. For example, api.METHOD([ 'post', 'put' ], '/some/path', (req,res)=> {})

This would be similar to the idea of an any() convenience method, but would restrict the allowable methods.

Like Express, create a way to set the location header:

res.location('/foo/bar');
res.location('http://example.com');
res.location('back');

This would NOT set a status code or complete the request. A call to res.send() or some other execution ending call would need to be made.

This seems like too common of a use case to require middleware every time to accomplish this. The method below is one way of handling preflight requests now.

api.options('/*', function(req,res) {
  // Add CORS headers
  res.header('Access-Control-Allow-Origin', '*');
  res.header('Access-Control-Allow-Methods', 'GET, PUT, POST, DELETE, OPTIONS');
  res.header('Access-Control-Allow-Headers', 'Content-Type, Authorization, Content-Length, X-Requested-With');
  res.status(200).send({});
})

There are six response headers documented here: https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS

• Access-Control-Allow-Origin
• Access-Control-Expose-Headers
• Access-Control-Max-Age
• Access-Control-Allow-Credentials
• Access-Control-Allow-Methods
• Access-Control-Allow-Headers

These could easily be packaged in a configuration object when calling something like:

res.cors({ 
  origin: 'example.com', 
  methods: 'GET, PUT, POST, DELETE, OPTIONS', 
  headers: 'Content-Type, Authorization',
  maxAge: 84000
})

I'm not sure this is much shorter, but it could always come with defaults.

For most of my apps I'm extracting a bearer token from the authorization header. Might be nice to have this common use case canned. I'll also need to research some other common forms and see if those will be easy enough to implement as well.

As I was looking into a separate issue where my files are coming out base64 encoded, I was browsing the sendFile() code. I noticed that when retrieving a file from S3, it retrieves the whole object before sending it back to the requester:

let data = await S3.getObject(params).promise()

This may be fine for smallish files, but for large files, this could potentially consume a TON of memory. A more efficient approach is to start sending data back to the client as soon as you start getting it from S3.

With S3 you can get a Node.js stream as so:

s3.getObject(params).createReadStream()

See also: https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/requests-using-stream-objects.html

You can send cache-control, last-modified and expires headers with static files, but not with routes. There should be convenience methods to add these headers to JSON, HTML and other text outputs.

From https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-context.html. These values are available via the req._context object, but some of these could be baked in.

exports.handler = function(event, context, callback) {
    console.log('remaining time =', context.getRemainingTimeInMillis());
    console.log('functionName =', context.functionName);
    console.log('AWSrequestID =', context.awsRequestId);
    console.log('logGroupName =', context.log_group_name);
    console.log('logStreamName =', context.log_stream_name);
    console.log('clientContext =', context.clientContext);
    if (typeof context.identity !== 'undefined') {
        console.log('Cognito identity ID =', context.identity.cognitoIdentityId);
    }    
};

I've implemented support for HEAD requests in v0.5 (#19), but I'd like to add a convenience method to override specific calls.

Saw this in a few places and it could be used to solve some use cases. Devs would need to handle the METHOD on there own.

Add a getHeader() method on the RESPONSE object to retrieve all current headers OR a specific header if passed as an argument, e.g res.getHeader('Content-Type').

This should ignore case as well.

Similar to express, being able to specify multiple middleware handlers using a single use statement could be useful.

app.use(mn1, mn2);

app.use('/somepath', mn1, mn2)

Wildcard routes are useful mostly for OPTIONS calls, which are necessary for preflight CORS checks. If you are splitting your app into multiple functions, being able to wildcard just certain paths would be very useful.

Since this is meant for Lambda, it should load files from an alternate source, such as S3. The only file that might make sense to host in the Lambda function itself is the favicon.ico, since browsers will automatically request that.

In the case where you are protecting file access, a redirect to an S3 bucket might make more sense. I guess it is possible to load the file from S3 and transfer through Lambda and APIG, but that would be a lot of extra overhead.

If you don't explicitly return an error (e.g. using throw), then subsequent res.send() requests will leak into the final response.

This might be as simple as checking the response._state in the main _callback method.

Add support for other web framework middleware.

Like Express, this should redirect to the URL derived from the specified path, with specified status, a positive integer that corresponds to an HTTP status code. If not specified, status defaults to “302 “Found”.

res.redirect('/foo/bar');
res.redirect('http://example.com');
res.redirect(301, 'http://example.com');
res.redirect('../login');

A call to this method would end the execution and return the response to the client. There may need to be some default language here.

Hey :) Cool library!

Would it be possible to use https://github.com/coopernurse/caddy-awslambda instead of API Gateway in front of lambda?

I guess some work would have to be done, bc the request envelope formats for the two "proxies" differ.

Do you think this is easy to integrate? I would be happy to supply a pull request with all needed changes/extensions 👍

Lambda API isn't designed for loading large files since it has to redirect them through API Gateway and there is no streaming support. So instead, it makes more sense to redirect the user to an S3 bucket. The SDK supports getSignedUrl, which will use Lambda's S3 credentials to generate a temporary link to an S3 file. I think this would be a nice convenience feature.

Documented here: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#getSignedUrl-property

First of all, just wanted to say nice work! It's nice to have such a streamlined library for use within a Lambda!

I'm trying to use the library for some downloading of binary files from S3... usually Zip file. This was incredibly easy to get set up! And yes, I did follow the instructions to enable binary mode by adding / in API Gateway.

The problem I am having is that I can't unzip the file after I have downloaded it as it is still base64 encoded. I can decode it manually and then unzip it, so I do know that this is the problem.

I'm trying to understand the reason for the base64 encoding... we should be able to download non-encoded files. It seems that the reason this encoding was performed is some requirement in API Gateway, is that the case? Maybe your Integration has CONVERT_TO_BINARY set?

I'm trying to read the AWS documentation to figure out if that needs to be set... Per that link:

"For API Gateway to pass binary payloads, you add the media types to the binaryMediaTypes list of the RestApi resource OR set the contentHandling properties on the Integration and the IntegrationResponse resources."

So, does CONVERT_TO_BINARY really need to be set? And, if not, should "lambda-api" be encoding it as Base64?

Now that Lambda supports v8.10, we should be able to declare our main handler like this:

exports.handler = async (event, context) => {
  // Run the request
  return api.run(event, context)
}

This should be optional for backwards compatibility.

I'm thinking this may be an async issue with dynamo db, but I'm not sure. Here is the offending function:

import shortid from 'shortid';
import dayjs from 'dayjs';

import dynamoDb from './db';
import validate from './schemas';

export const create = (req, res) => {
    const data = req.body;

    const validation = validate(data);
    if (validation.error) {
        console.error('Validation error in creating project resource.');
        res.status(400).json({
            message: 'Validation error.',
            error: validation.error
        });
    }

    const {title, subject, abstract, plan, status} = data;
    const timestamp = dayjs().toISOString(); 
    const project = {
        id: shortid.generate(),
        timestamp,
        title,
        subject,
        abstract,
        plan,
        status
    };

    const params = {
        TableName: process.env.DYNAMODB_TABLE,
        Item: project
    };

    // write the todo to the database
    dynamoDb.put(params, error => {
        // handle potential errors
        if (error) {
            console.error(error);
            res.status(501).json({message: 'error', error});
        }

        res.status(201).json(params.Item);
    });
};

export default create;

(using babel/webpack for import/export syntax)

main app code is here:

import LambdaApi from 'lambda-api';

import list from './list';
import create from './create';

const app = LambdaApi({base: 'aegis'});

app.use((req, res, next) => {
    res.cors();
    next();
});

app.get('/projects', list);
app.post('/projects', create);

// Default Options for CORS preflight
app.options('/*', (req, res) => {
    res.status(200).json({});
});

export const listener = (event, context, callback) => {
    context.callbackWaitsForEmptyEventLoop = false;
    app.run(event, context, callback);
};

For some reason, objects are still getting stored in the database even if they fail validation (and the correct 400 response is sent back). The request is supposed to stop there when you use the res.json() callback, correct? used the sample api as a starting point for what it's worth.

Any ideas?

sls invoke is useful for testing our API one request at a time.

It would be nice to spin up a local version of the API on localhost to enable us to develop our frontend apps against it, and to more easily run integration and E2E tests.

I've run some preliminary speed test and I've been consistently clocking Lambda functions running Lambda API at less that 0.68 ms for total execution time. This is will several configured routes. I'd like to set up some benchmark tests to compare this to other popular frameworks running on top of AWS Lambda.

Add a format() method to the RESPONSE object that can negotiate the response based on the Accept header. Similar to https://expressjs.com/en/api.html#res.format

Simple callback wrapper for JSONP requests. The callback name will default to callback, but can be changed by using the following querystring parameter: ?callback=foo.

After the latest release, the README.md file is getting a bit unwieldy. There are a lot of features now, and I like to give more examples and show potential use cases.

I guess I could start using JSDoc and create it from that, rather than trying to maintain the docs separately. Suggestions would be welcome.

Hi,

Tried setting up a simple example and had trouble with middleware. Can't get middleware example from documentation to work:

api.use((req,res,next) => {
    if (req.headers.authorization === 'some value') {
        req.authorized = true
        next() // continue execution
    } else {
        res.status(401).error('Not Authorized')
    }
})

What happens is when I should see a 401 response I instead see a 502 response. I debugged it for hours and could only determine that it seems like a 2nd response/body is being set or it's going through the final callback a second time by mistake. I think the 502 comes from API Gateway not parsing the response. I was testing using API Gateway Local Runner on Cloud9. If I move the "res.error" line of code into a route the proper 401 response is returned. Until we get this fixed, that's my temporary workaround. Any ideas?

Its not uncommon to use TypeScript over top of node.js and being able to reference a TypeScript model would help a ton when using lambda-api in that context.

Lost quite some time on this so I think it would be a good idea to put in the docs.. let me know if you want me to create a pull request..

The problem is when you have a connection pool outside your lambda handler code (I use mongoose/mongodb).. in that case the event loop never ends and your function always time out.
In that case you must add the following code inside your handler:

context.callbackWaitsForEmptyEventLoop = false;

Complete code:

'use strict';
//lambda-api app
const app = require('./app');
//db initialization module
const db = require('./app/db_init');

//Initialize database connection pool
db();

module.exports.app = (event, context, callback) => {
  context.callbackWaitsForEmptyEventLoop = false; //don't wait for empty event loop to return
  app.run(event, context, callback);
};

More info about this property here: https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-context.html

Another option is to use the context.done and context.fail methods, they always return immediately, but I think people are moving more to using callback instead.

Method to clear cookie by name and optional arguments.

res.clearCookie('name', { path: '/admin' });

This method does NOT send a response to the client until an ending call is made.

Node 8.10 runtime is now available in Lambda! https://aws.amazon.com/blogs/compute/node-js-8-10-runtime-now-available-in-aws-lambda/

I'd really like to refactor to use async / await instead of Bluebird so that I can remove ALL dependencies from Lambda API. I think we're early enough into this project to to require the 8.10 runtime in our Lambda functions.

I'm currently using a Promise.each() to serialize middleware, so I'll need to figure out a new approach for that. However, this might create an opportunity to allow middleware to optionally process asynchronously. I'll have to think about this some more.

When accessing API Gateway directly through a web browser, you get an annoying 403/404 error depending on how you've configured your proxy. This may be a non-issue for API use cases, but if you are using API Gateway to return HTML content, then it will keep happening.

Maybe there should be a default route for /favicon.ico responses that simple returns a blank image. It could be overridden with a custom path as well. This might benefit from the sendFile functionality.

The implementation is fairly simple on an Apache server, so I think adding a Link header might work in an APIG/Lambda environment. Maybe. Any thoughts on this are welcome.

Provide a logging mechanism to help developers control logging through logging levels as well as provide additional context information about each log entry.

It should support (at a minimum): info, error, debug, fatal, warn

Logs should be in JSON format and contain timestamp, route info, etc.

I am implementing CORS through your example of the options/wildcard implementation. I am sure I am doing something wrong, but I believe this should work.

const api = require('lambda-api')()

// Options
api.options('/*', (req,res) => {
    // Add CORS headers
    res.header('Access-Control-Allow-Origin', '*');
    res.header('Access-Control-Allow-Methods', 'GET, PUT, POST, DELETE, OPTIONS');
    res.header('Access-Control-Allow-Headers', 'Content-Type, Authorization, Content-Length, X-Requested-With');
    res.status(200).send({});
  })

// Services
api.register(require('./controllers/slink'), { prefix: '/api/user' });
api.register(require('./controllers/cart'), { prefix: '/api/cart' });
api.register(require('./controllers/price'), { prefix: '/api/price' });

module.exports.handler = (event, context, callback) => {
    api.routes(true);
    api.run(event, context, callback);
}

Any thoughts?

There are request parsing differences with CloudFront as the event is structured differently:
https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-event-structure.html

It would be useful to have the ability to also process HEAD requests as an alternative to GET if you're looking for just the headers. While I suppose that we could create a head() convenience method, it seems like a lot of duplicate work.

Without being overly opinionated, Lambda API could respond to any HEAD request so long as there was a corresponding GET route for it and then short circuit the response to omit the body. If there were a specific response for HEAD requests, those could always be overwritten by an explicit head() route, although it should still probably enforce the response short circuiting.

api.register(require('./routes/v1/products'))

The above throws an error when no object is passed in as the second parameter. This should be defaulted so it isn't undefined.

The .send() method is returning the correct route data but isn't terminating execution of the script.

If the Content-Type header is passed in all lowercase (or any other case that doesn't match exactly) with application/x-www-form-urlencoded, it fails to auto parse the form encoded body.

The header needs to be converted to lowercase for comparison.

The majority of the work would still need to be handled by the developer, but there could be some convenience methods that made this easier to implement.

Hi,
I'm trying to send a 404 status as response of an endpoint using async and await

    api.get('/:id', async (req, res) => {
        res.status(404)
    })

but I'm receiving a 200 response.

Using the example from the readme I'm getting a 500 error

    api.get('/users', (req,res) => {
        res.status(401).error('Not Authorized')
    })

If I use another way to write the error it works:

    api.get('/:id', async (req, res) => {
        res.error(404, "Not Found")
    })

The following should be supported to allow for more complex URL patterns.

api.get('/example/:lat-:lng/radius/:r', (req,res) => { })

This method would set the HTTP Set-Cookie header with the options provided. There would need to be some defaults as well.

res.cookie('name', 'tobi', { domain: '.example.com', path: '/admin', secure: true });
res.cookie('rememberme', '1', { expires: new Date(Date.now() + 900000), httpOnly: true });

This method would NOT send the response to the browser until something like res.send() is called.

While working on cache control for the sendFile() method, I did some additional research into the Etag header. Seems easy enough to implement and could be useful for saving data transfer fees in Lambda.

I like how Fastify provides a register() method to handle route prefixing and a convenience method for loading routes from an external file. Right now you can add external routes by exporting routes with a reference to the API instance. This is sort of messy and I like the idea of adding the convenience method.