Code Monkey home page Code Monkey logo

amazon-location-service-serverless-address-enrichment's Introduction

Serverless Address Enrichment with Amazon Location Service

This repository accompanies a post from the AWS Compute Compute Blog Enriching addresses with AWS Lambda and the Amazon Location Service. The repository contains a SAM tempalte for deploying a Serverless Address Enrichment pipeline using Amazon S3, AWS Lambda, and Amazon Location Service.

Highlevel Architecture

Screen Shot 2021-12-09 at 12 08 50 PM

  1. The Scatter Lambda function takes a data set from the S3 bucket labeled input and breaks it into equal sized shards.
  2. The Process Lambda function takes each shard from the pre-processed bucket and performs Address Enrichment in parallel calling the Amazon Location Service Places API
  3. The Gather Lambda function takes each shard from the post-processed bucket and appends them into a complete dataset with additional address information.

Deploying the Project

Prerequistes:

To use the SAM CLI, you need the following tools:

This Sample Includes:

  • template.yaml: Contains the AWS SAM template that defines you applications AWS resources, which includes a Place Index for Amazon Location Service
  • scatterfunction/: Contains the Lambda handler logic behind the scatter function and its requirements
  • 2waygeocoderfunction/: Contains the Lambda handler logic for the processor function which calls the Amazon Location Service Places API to perform address enrichment
  • gatherfunction/: Contains the Lambda handler logic for the gather function which appends all of processed data into a complete dataset

Deploy the Sam-App:

  1. Use git clone https://github.com/aws-samples/amazon-location-service-serverless-address-validation to clone the repository to your environment where AWS SAM and python are installed.
  2. Use cd ~/amazon-location-service-serverless-address-validationto change into the project directory containing the template.yaml file SAM uses to build your application.
  3. Use sam build to build your application using SAM. You should see:

Screen Shot 2021-12-13 at 3 02 51 PM

  1. Use sam deploy --guided to deploy the application to your AWS account. Enter custom values for the application parameters.

    • NOTE: be sure to follow S3 naming conventions by providing globally unique names for your s3 buckets such as input-YOURNAME raw-YOUR-PHONE-NUMBER or destination-YOUR-AWS-ACCOUNT-ID. See below for an example.

Screen Shot 2021-12-13 at 3 15 11 PM

Testing the Application

Download the below samples locally, unzip the files, and upload the CSV to your input S3 bucket to trigger the adddress enrichment pipeline.

Geocoding: City of Hartford, CT Business Listing Dataset

Reverse Geocoding: Miami Housing Dataset

Security

See CONTRIBUTING for more information.

License

This library is licensed under the MIT-0 License. See the LICENSE file.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.