Code Monkey home page Code Monkey logo

lambdacg's Introduction

Lambda Composition Gateway

Imagine multiple software development teams: one working on a frontend application, and several others working on (backend) services that supply the queries and commands that the frontend application uses.

If the frontend were to invoke the services directly, this would mean a tight coupling between them; not only between frontend and backend, but possibly also between the backend services. Teams would need to communicate a lot, and from time to time they would depend on one of the other teams to implement part of new feature.

This repository contains the Lambda Composition Gateway (LambdaCG), a proof of concept that decouples software components like these.

Building and testing

Required tool: nodejs (including npm).

Go to the repository root directory.

Before anything else:

npm ci

To run all the unit tests (again, at the root of the repo):

npm test

to run all the component tests (also see next section):

npm run component-test

This repository has a so-called monorepo setup, which means each of the three projects can also be independently built, tested and run.

Testing AWS functionality

The component tests are executed in the cloud, on AWS.

In order run the component tests, you need an AWS account. If you do not have an aws account configured, these tests will be skipped.

If you want to run these tests by yourself, you can do the following:

  1. Fork this repository
  2. Follow instructions on how to create an AWS account here.
  3. Download the AWS command-line interface and then configure it.
  4. Deploy the testing configuration by running the infra/deploy-github-lambdacg-testing-config.sh script.
  5. (Optional): Enable github actions running these tests on your repository by installing a secret with the name AWS_TESTING_ROLE, containing the ARN of the GithubLambdaCgTestingRole role that was deployed by the script.

NOTE:

  • All the AWS tests that run will create some infrastructure and clean up again after they finish.
  • Note that running tests against your AWS account will cost you (a little bit) of money.

Architecture

LambdaCG consists of three components:

The following describes an example of how they could cooperate with 3 services.

LambdaCG overview

There are 3 services, maintained by 3 teams: teams/services Blue, Purple and Red. Each of the services has the need to process (parts of) requests that arrive at the AppSync graphQL API.

  1. Each team/service publishes a handler for well-defined requests to an S3 bucket. Each of these handlers implement the contract defined by lambdacg-contract.

    Note that a handler could insert something into a database (Blue handler), or send a request to an API (Purple handler); it could also be just a bit of logic, that doesn't need any communication to any backend system (Red service).

  2. The S3 bucket is configured to trigger the lambdacg-updater lambda. In response, that lambda downloads the published handlers, packages them and deploys them into the lambdacg-resolver.

  3. The lambdacg-resolver has available all the handlers that were published by the services. It doesn't know where these handlers came from or what they do; it only knows that they all implement lambdacg-contract. Because of this, it knows how to invoke them and how to interpret their results, and combine them into a result that can be sent back through AppSync.

Status

Status of this PoC is as follows:

  • lambdacg-contract: Done.
  • lambdacg-resolver: Code complete, full unit test coverage.
  • lambdacg-updater: Code complete, full unit test coverage, component test in place.
  • infra: Cloudformation templates to deploy the whole thing, not started yet.

lambdacg's People

Contributors

rkamphorst avatar dependabot[bot] avatar

Watchers

 avatar

lambdacg's Issues

Updater: send the list in 2 places: assed handlerFactories.json and env var HANDLER_FACTORIES

With #4, updater changed the way the list of handler factory modules are passed to the resolver: instead of storing the list in a file (handlerFactories.json), the list is now set as a string (comma-separated list of module names) in the environment variable HANDLER_FACTORIES on the target lambda. The reasoning was that environment variables are easier to change after the function has been deployed.

The drawback is that the code upload happens first, after which the lambda updates itself. Only then, the configuration change with the environment variables can occur.

Therefore, we need the following things to occur.

  • Make the updater store the list of module names in a file again; but this time, use the 'assets' utility module that was developed for updater. So the file will be assets/handlerFactories.json
  • Let resolver read this asset in order to get the list of module names. Do NOT duplicate the code in assets.ts; instead, create a separate utilities project (lambdacg-utils) that contains the shared utility code of lambdacg projects.
  • Resolver should still read the HANDLER_FACTORIES environment variable. If it is present, only the modules that are in both HANDLER_FACTORIES and assets/handlerFactories.json should be enabled
  • Resolver should also read the DISABLE_HANDLER_FACTORIES environment variable. If it is present, the modules that are in both HANDLER_FACTORIES and assets/handlerFactories.json should be DISabled. If a module is in both HANDLER_FACTORIES and DISABLE_HANDLER_FACTORIES, the latter wins.

Eliminate resolver dependency on lodash

The lodash dependency in itself takes 300kb. This is overkill especially since it is needed in three places. Without lodash, the unzipped size of the package is 15kb

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.