Code Monkey home page Code Monkey logo

dynamodb-datagenerator's Introduction

dynamoDB-dataGenerator

This repository was created in response to my requirement of loading in a reasonably-large data set to Amazon DynamoDB. The data in question is a number of IP Subnets which I will use as a basic IPAM.

  1. Firstly I needed to create a shell script to generate my JSON file into the format required by DynamoDB.
  2. Next I needed to populate my DB with the JSON data in the form of: aws dynamodb batch-write-item --request-items file://subnets.json

json-generator.sh creates the JSON in the format required by AWS CLI. However I ran into an issue - the AWS CLI only supports adding up to 25 items at a time!

This lead me to find the Boto3 Python library to interact with DynamoDB: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/dynamodb.html

dynamodb_batchWriteFromJson.py was created to take a JSON named subnets.json and write these values, one at a time, to my DB table named SubnetManager

The JSON format needed to be different, and so I created a new JSON generator json-generator-boto.sh

dynamodb-datagenerator's People

Contributors

scottymcandrew avatar

Watchers

 avatar

dynamodb-datagenerator's Issues

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.