Code Monkey home page Code Monkey logo

cosmos-shuttle's Introduction

Cosmos Shuttle

Cosmos Shuttle is a simple data import/export tool for Cosmos DB.

Building the project

To build the project into the tool executable:

  1. Clone this repository
  2. From a command line terminal, change to .../cosmos-shuttle/CosmosShuttle
  3. Run build command: dotnet publish -c Release -r win-x64 --self-contained
  4. Locate the produced CosmosShuttle.exe file under .../bin/Release/net7.0/win-x64/publish

Running the tool

Given a built CosmosShuttle.exe executable, you can run it to perform an import or export.

Obtaining a connection string

The tool authenticates with a Cosmos account connection string. This can be found in Azure Portal > Cosmos account > Settings > Keys.

When exporting, a read-only key will suffice. When importing, a read-write key is required.

Export data from Cosmos DB

CosmosShuttle.exe export --db <DATABASE_NAME> --container <CONTAINER_NAME> --connection "<CONNECTION_STRING>"

where:

  • DATABASE_NAME: Source Cosmos DB database
  • CONTAINER_NAME: identifier of a source container from which to export all items
  • CONNECTION_STRING: connection string value (Read-only or Read-Write)

If successful, a JSON file is created representing all the items in the container.

Import data into Cosmos DB

CosmosShuttle.exe import --source <SOURCE_FILE> --db <DATABASE_NAME> --container <CONTAINER_NAME> --batchsize <BATCH_SIZE> --connection "<CONNECTION_STRING>"

Emulator example:

CosmosShuttle.exe import --source export.json --db contoso-db --container data --connection emulator

Available parameters:

  • after: Export items with _ts timestamp after this value. Can be UNIX seconds like 1670358000 or ISO 8601 timestamp like 2022-12-25T08:00
  • batchsize: Number of upsert operations to perform in parallel batches. 1 to 500 allowed. Default: 25
  • camelcase: Specifying true converts all items to have camelCased property keys during import
  • connecton: connection string value (Read-Write). May use emulator as shortcut for local emulator conneciton string.
  • container: identifier of a target container into which to import all items
  • db: Target Cosmos DB database
  • logging: Controls verbosity of output. Levels: info | verbose
  • source: Path to the JSON file exported by this tool. This data will be imported into the target container.
  • timeout: Number of seconds to wait for responses from Cosmos DB. Default: 120

If successful, all items in the source file will be created or updated in the target container.

The tool uses an upsert operation based on the item's id value. If an item already exists in the target container with the same id, it will be overwritten with the source item. Otherwise, it will be created.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.