Code Monkey home page Code Monkey logo

horgh-replicator's Introduction

Mysql binlog replicator

Site

See full docs here

Master types

  • MySQL

Slave types

  • MySQL
  • PostgreSQL
  • Yandex ClickHouse
  • HP Vertica

Quick start

See quick start tutorial here

Testing

  • Copy src/.env.dist to src/.env and set credentials.
  • Configure your my MySQL master as examples/master/mysql.conf. Don't forget to set binlog_do_db=<master_db_name> and restart MySQL service.
  • Create databases and tables as examples/sql/.
  • Start Docker as make start-dev
  • Run as cd src and go run main.go listen in docker container.
  • Copy examples/configs/user.json and examples/configs/post.json to src/system/configs
  • Execute cd src and go run main.go load

Add tables to replicator

  • Use create-model <table> to create json config for your table.
  • Create table on slave.
  • Set <table> to TABLE param in src/.env
  • Use build-slave to copy table data from master and set start position of log for table listener.

Custom handlers for field value

  • Create plugins/user/<plugin_name>/handler.go like plugins/system/set_value/handler.go
  • Execute go build -buildmode=plugin -o plugins/user/<plugin_name>/handler.so plugins/user/<plugin_name>/handler.go
  • Add to field description in your src/system/configs/<model>.json
"beforeSave": {
  "handler": "user/<plugin_name>",
  "params": [
    "***"
  ]
}
System handlers
  • If you want to set custom field value use system/set_value as handler param. Don't forget to set params: ["<value>"]

Tools

  • set-position <table> <binlog_name> <binlog_position> set start position of log for table listener
  • load start loader for replication testing (for default tables user and post)
  • create-model <table> create model json-file by master table structure
  • build-slave create master table dump, restore this dump in slave, set start position of log for table listener
  • destroy-slave truncate table, set empty position of log for table listener

Modes

  • Prod mode: build app and execute listener.

    Use make build-prod and make start-prod to start and make stop-prod to stop.

  • Dev mode: provides the opportunity for manual start and debug.

    Use make start-dev to start and make stop-dev to stop.

horgh-replicator's People

Contributors

larsnovikov avatar

Watchers

James Cloos avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.