Code Monkey home page Code Monkey logo

momy's Introduction

Build Status NPM Status Codecov Status

Momy

Momy is a simple cli tool for replicating MongoDB to MySQL in realtime.

  • Enable SQL query on data in NoSQL database
  • Enable to be accessed by Excel / Access

Momy

Installation

$ npm install -g momy

or install it within the project locally:

$ npm install --save momy

Preparation

MongoDB

Momy uses Replica Set feature in MongoDB. But you don't have to replicate between MongoDB actually. Just follow the steps below.

Start a new mongo instance with no data:

$ mongod --replSet "rs0" --oplogSize 100

Open another terminal, and go to MongoDB Shell:

$ mongo
....
> rs.initiate()

rs.initiate() command prepare the collections that is needed for replication.

MySQL

Launch MySQL instance, and create the new database to use. The tables will be created or updated when syncing. You'll see mongo_to_mysql, too. This is needed to store the information for syncing. (don't remove it)

Configuration

Create a new momyfile.json file like this:

{
  "src": "mongodb://localhost:27017/dbname",
  "dist": "mysql://root:password@localhost:3306/dbname",
  "prefix": "t_",
  "case": "camel",
  "collections": {
    "collection1": {
      "_id": "number",
      "createdAt": "DATETIME",
      "field1": "number",
      "field2": "string",
      "field3": "boolean",
      "field4.subfield": "string"
    },
    "collection2": {
      "_id": "string",
      "createdAt": "DATETIME",
      "field1": "number",
      "field2": "string",
      "field3": "boolean",
      "field4": "TEXT"
    }
  }
}
  • src: the URL of the MongoDB server
  • dist: the URL of the MySQL server
  • prefix: optional prefix for table name. The name of the table would be t_collection1 in the example above.
  • fieldCase: optional. snake or camel. See the section below.
  • exclusions: optional. Chars or a range of chars to exclude: "\uFFFD"
  • inclusions: optional. Chars or a range of chars to include: "\u0000-\u007F"
  • collections: set the collections and fields to sync

_id field is required for each collection and should be string or number.

Field names and types

"<field_name>": "<field_tipe>"

or, field_name could be dot-concatenated:

"<field_name>.<sub_name>": "<field_tipe>"

For example, if you have { a: { b: { c: 'hey!' } } } then "a.b.c": "string"

Currently these native types are supported:

  • BIGINT
  • TINYINT
  • VARCHAR
  • DATE
  • DATETIME
  • TIME
  • TEXT

There're also some aliases:

  • number => BIGINT
  • boolean => TINYINT
  • string => VARCHAR

Field name normalization: fieldCase

Some system like Microsoft Access don't allow dot-concatenated field names, so address.street will cause an error. For such a case, use fieldCase:

  • snake: address.street --> address_street
  • camel: address.street --> addressStreet

Note: if you set fieldCase value, the name of _id field will change into id without _, too.

Usage

At the first run, we need to import all the data from MongoDB:

$ momy --config momyfile.json --import

Then start the daemon to streaming data:

$ momy --config momyfile.json

or

$ forever momy --config momyfile.json

For contributors

Before testing, install MySQL into your environment. Then, create a new database:

$ mysql -uroot -e 'create database momy;'

To run the tests:

$ npm test

To test the code manually, start mongodb and momy:

$ npm run pretest
$ npm run try

Do something, then kill momy and stop mongodb:

$ npm run posttest

License

MIT

This library was originally made by @doubaokun as MongoDB-to-MySQL and rewritten by @cognitom.

momy's People

Contributors

cognitom avatar chemitaxis avatar doubaokun avatar odede-perfecto avatar vagr9k avatar ardinusawan avatar tmvanetten avatar

Watchers

James Cloos avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.