toughiq / docker-mariadb-cluster Goto Github PK

Dockerized MariaDB Galera Cluster

License: GNU General Public License v2.0

Shell 75.83% Dockerfile 24.17%

docker-mariadb-cluster's Introduction

docker-mariadb-cluster

Version 2 Dockerized Automated MariaDB Galera Cluster

Version 2 is the advanced branch and is featured on DockerHub as latest from now on. Old version 1.0 can be found here: https://github.com/toughIQ/docker-mariadb-cluster/tree/v1. To get V1.0 Docker images, just docker pull toughiq/mariadb-cluster:1.0

The idea of this project is to create an automated and fully ephemeral MariaDB Galera cluster. No static bindings, no persistent volumes. Like a disk RAID the data gets replicated across the cluster. If one node fails, another node will be brought up and the data will be initialized.

Consider this a POC and not a production ready system!

Built for use with Docker 1.12.1+ in Swarm Mode

WORK in Progress!

See Issues for known problems and Wiki for notes and ideas.

Setup

Init Swarm Nodes/Cluster

Swarm Master:

docker swarm init

Additional Swarm Node(s):

docker swarm join <MasterNodeIP>:2377 + join-tokens shown at swarm init

To get the tokens at a later time, run docker swarm join-token (manager|worker)

Create DB network

docker network create -d overlay mydbnet

Init/Bootstrap DB Cluster

At first we start with a new service, which is set to --replicas=1 to turn this instance into a bootstrapping node. If there is just one service task running within the cluster, this instance automatically starts with bootstrapping enabled.

docker service create --name dbcluster \
--network mydbnet \
--replicas=1 \
--env DB_SERVICE_NAME=dbcluster \
toughiq/mariadb-cluster

Note: the service name provided by --name has to match the environment variable DB_SERVICE_NAME set with --env DB_SERVICE_NAME.

Of course there are the default MariaDB options to define a root password, create a database, create a user and set a password for this user. Example:

docker service create --name dbcluster \
--network mydbnet \
--replicas=1 \
--env DB_SERVICE_NAME=dbcluster \
--env MYSQL_ROOT_PASSWORD=rootpass \
--env MYSQL_DATABASE=mydb \
--env MYSQL_USER=mydbuser \
--env MYSQL_PASSWORD=mydbpass \
toughiq/mariadb-cluster

Scale out additional cluster members

Just after the first service instance/task is running with we are good to scale out. Check service with docker service ps dbcluster. The result should look like this, with CURRENT STATE telling something like Running.

ID                         NAME         IMAGE                    NODE    DESIRED STATE  CURRENT STATE           ERROR
7c81muy053eoc28p5wrap2uzn  dbcluster.1  toughiq/mariadb-cluster  node01  Running        Running 41 seconds ago

Lets scale out now:

docker service scale dbcluster=3

This additional 2 nodes start will come up in "cluster join"-mode. Lets check again: docker service ps dbcluster

ID                         NAME         IMAGE                    NODE    DESIRED STATE  CURRENT STATE               ERROR
7c81muy053eoc28p5wrap2uzn  dbcluster.1  toughiq/mariadb-cluster  node01  Running        Running 6 minutes ago       
8ht037ka0j4g6lnhc194pxqfn  dbcluster.2  toughiq/mariadb-cluster  node02  Running        Running about a minute ago  
bgk07betq9pwgkgpd3eoozu6u  dbcluster.3  toughiq/mariadb-cluster  node03  Running        Running about a minute ago

Create MaxScale Proxy Service and connect to DBCluster

There is no absolute need for a MaxScale Proxy service with this Docker Swarm enabled DB cluster, since Swarm provides a loadbalancer. So it would be possible to connect to the cluster by using the loadbalancer DNS name, which is in our case dbcluster. Its the same name, which is provided at startup by --name.

But MaxScale provides some additional features regarding loadbalancing database traffic. And its an easy way to get information on the status of the cluster.

Details on this MaxScale image can be found here: https://github.com/toughIQ/docker-maxscale

docker service create --name maxscale \
--network mydbnet \
--env DB_SERVICE_NAME=dbcluster \
--env ENABLE_ROOT_USER=1 \
--publish 3306:3306 \
toughiq/maxscale

To disable root access to the database via MaxScale just set --env ENABLE_ROOT_USER=0 or remove this line at all. Root access is disabled by default.

Check successful startup of Cluster & MaxScale

Execute the following command. Just use autocompletion to get the SLOT and ID.

docker exec -it maxscale.<SLOT>.<ID> maxadmin -pmariadb list servers

The result should report the cluster up and running:

-------------------+-----------------+-------+-------------+--------------------
Server             | Address         | Port  | Connections | Status              
-------------------+-----------------+-------+-------------+--------------------
10.0.0.3           | 10.0.0.3        |  3306 |           0 | Slave, Synced, Running
10.0.0.4           | 10.0.0.4        |  3306 |           0 | Slave, Synced, Running
10.0.0.5           | 10.0.0.5        |  3306 |           0 | Master, Synced, Running
-------------------+-----------------+-------+-------------+--------------------

docker-mariadb-cluster's People

Contributors

Stargazers

Watchers

Forkers

mtahiue mulderh barnumd hafeez3000 atchariya judypol nevalla junlapong haijunsu sysc0d murf butonic zx1986 ovanassche ngocngv dottgonzo ziguilp imjerrybao vqiu nekulin xingjiudong dockeraa hydracz limweb bluemutedwisdom ethan-2017 hitomidennis jay74jung ly798 animalcarpet agungsijawir zhangjianasp rtheobald yelmir kerrer tanzhiliangit wilsonliu78 piyapsri docker-craft streetracer48 foleymic mckatoo blue-cosmos droidwiki duong1989 tamasgal mortn comdata gurjindersingh wolvenspirit changsongyang cheeraccident moussdia andersonbr-oss hummans dsilabs renowncoder georg480 locvfx nikitasardov maorin flamesrunner jeffyoungl swisslearninghub itdataengsirim

docker-mariadb-cluster's Issues

Node not detected on maxscale

Hi
first of all thanks for really good instruction .

i have been using cluster on test environment and i'v done db cluster running . but when i use maxscale , it's show all servers down state .

is there any network issue or it's about maxscale ?

i can access nodes within ip on new network created and still have access to maxscale .

Pulse check

It's been a minute since this repo saw a commit. As a result, I have two questions:

Who's using this in production with something like wordpress-swarm?
Other than persistence, are there any glaring needs left to call this product stable?

This is not mariadb detected

If you connect to this cluster server with mariadb using on navicat 11.1.12.
80270006: This is not MariaDB server.

why? and cant add tag mariadb configuration for my.cnf?

Using in production.

Hi,

I'm trying to find reliable solution to run SQL cluster in docker swarm. So it turns out that this project provides probably the easiest way to do that. But as mentioned in README file, it's not ready for production. So I would like to know, if there are any other projects like this you will recommend and what issues will I face if I use this project in production?

MariaDB crashes on multi-node cluster

If you run the cluster on a single node, just initialized by docker swarm init, everything works as expected.
But as soon as the cluster consists of more than one node, connected by the created overlay network, the dbcluster instances, which get started on different hosts than the bootstrap container, will crash at/after syncing with the bootstrap server.

Thanks

Upgrade to mariaDB10.4

Heyho,

your project seems fine and works quite well for me but can you maybe update the project to work with mariaDB 10.4, would be awesome to seem an ongoing maintenace here 😄
Currently 10.3 is in use and there was not update since 8 month on dockerHub: https://hub.docker.com/r/toughiq/mariadb-cluster/dockerfile

Regards

Service fails on startup

First of all, thanks for your work. If this one is running, it would be a very nice piece of my docker toolkit. But actually use with the following docker-compose file fails

version: "3"
services:

  database:
    image: toughiq/mariadb-cluster:10.1
    environment:
      DB_SERVICE_NAME: database
      MYSQL_ROOT_PASSWORD: 12345678
      MYSQL_DATABASE: test
      MYSQL_USER: test
      MYSQL_PASSWORD: 12345678

running docker-compose up database gives me the following output

database_1  | Initializing database
database_1  | 2017-12-14 11:31:32 139802196658112 [Note] /usr/sbin/mysqld (mysqld 10.1.29-MariaDB-1~jessie) starting as process 64 ...
database_1  | 2017-12-14 11:31:32 139802196658112 [Note] InnoDB: Using mutexes to ref count buffer pool pages
database_1  | 2017-12-14 11:31:32 139802196658112 [Note] InnoDB: The InnoDB memory heap is disabled
database_1  | 2017-12-14 11:31:32 139802196658112 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
database_1  | 2017-12-14 11:31:32 139802196658112 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
database_1  | 2017-12-14 11:31:32 139802196658112 [Note] InnoDB: Compressed tables use zlib 1.2.8
database_1  | 2017-12-14 11:31:32 139802196658112 [Note] InnoDB: Using Linux native AIO
database_1  | 2017-12-14 11:31:32 139802196658112 [Note] InnoDB: Using SSE crc32 instructions
database_1  | 2017-12-14 11:31:32 139802196658112 [Note] InnoDB: Initializing buffer pool, size = 256.0M
database_1  | 2017-12-14 11:31:32 139802196658112 [Note] InnoDB: Completed initialization of buffer pool
database_1  | 2017-12-14 11:31:32 139802196658112 [Note] InnoDB: The first specified data file ./ibdata1 did not exist: a new database to be created!
database_1  | 2017-12-14 11:31:32 139802196658112 [Note] InnoDB: Setting file ./ibdata1 size to 12 MB
database_1  | 2017-12-14 11:31:32 139802196658112 [Note] InnoDB: Setting log file ./ib_logfile101 size to 48 MB
database_1  | 2017-12-14 11:31:32 139802196658112 [Note] InnoDB: Setting log file ./ib_logfile1 size to 48 MB
database_1  | 2017-12-14 11:31:33 139802196658112 [Note] InnoDB: Renaming log file ./ib_logfile101 to ./ib_logfile0
database_1  | 2017-12-14 11:31:33 139802196658112 [Warning] InnoDB: New log files created, LSN=45781
database_1  | 2017-12-14 11:31:33 139802196658112 [Note] InnoDB: Doublewrite buffer not found: creating new
database_1  | 2017-12-14 11:31:33 139802196658112 [Note] InnoDB: Doublewrite buffer created
database_1  | 2017-12-14 11:31:33 139802196658112 [Note] InnoDB: 128 rollback segment(s) are active.
database_1  | 2017-12-14 11:31:33 139802196658112 [Warning] InnoDB: Creating foreign key constraint system tables.
database_1  | 2017-12-14 11:31:33 139802196658112 [Note] InnoDB: Foreign key constraint system tables created
database_1  | 2017-12-14 11:31:33 139802196658112 [Note] InnoDB: Creating tablespace and datafile system tables.
database_1  | 2017-12-14 11:31:33 139802196658112 [Note] InnoDB: Tablespace and datafile system tables created.
database_1  | 2017-12-14 11:31:33 139802196658112 [Note] InnoDB: Waiting for purge to start
database_1  | 2017-12-14 11:31:33 139802196658112 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.36-82.2 started; log sequence number 0
database_1  | 2017-12-14 11:31:33 139801361643264 [Note] InnoDB: Dumping buffer pool(s) not yet started
database_1  | 2017-12-14 11:31:36 140641052444608 [Note] /usr/sbin/mysqld (mysqld 10.1.29-MariaDB-1~jessie) starting as process 92 ...
database_1  | 2017-12-14 11:31:36 140641052444608 [Note] InnoDB: Using mutexes to ref count buffer pool pages
database_1  | 2017-12-14 11:31:36 140641052444608 [Note] InnoDB: The InnoDB memory heap is disabled
database_1  | 2017-12-14 11:31:36 140641052444608 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
database_1  | 2017-12-14 11:31:36 140641052444608 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
database_1  | 2017-12-14 11:31:36 140641052444608 [Note] InnoDB: Compressed tables use zlib 1.2.8
database_1  | 2017-12-14 11:31:36 140641052444608 [Note] InnoDB: Using Linux native AIO
database_1  | 2017-12-14 11:31:36 140641052444608 [Note] InnoDB: Using SSE crc32 instructions
database_1  | 2017-12-14 11:31:36 140641052444608 [Note] InnoDB: Initializing buffer pool, size = 256.0M
database_1  | 2017-12-14 11:31:36 140641052444608 [Note] InnoDB: Completed initialization of buffer pool
database_1  | 2017-12-14 11:31:36 140641052444608 [Note] InnoDB: Highest supported file format is Barracuda.
database_1  | 2017-12-14 11:31:36 140641052444608 [Note] InnoDB: 128 rollback segment(s) are active.
database_1  | 2017-12-14 11:31:36 140641052444608 [Note] InnoDB: Waiting for purge to start
database_1  | 2017-12-14 11:31:36 140641052444608 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.36-82.2 started; log sequence number 1616697
database_1  | 2017-12-14 11:31:36 140640235026176 [Note] InnoDB: Dumping buffer pool(s) not yet started
database_1  | 2017-12-14 11:31:39 140338210944960 [Note] /usr/sbin/mysqld (mysqld 10.1.29-MariaDB-1~jessie) starting as process 121 ...
database_1  | 2017-12-14 11:31:39 140338210944960 [Note] InnoDB: Using mutexes to ref count buffer pool pages
database_1  | 2017-12-14 11:31:39 140338210944960 [Note] InnoDB: The InnoDB memory heap is disabled
database_1  | 2017-12-14 11:31:39 140338210944960 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
database_1  | 2017-12-14 11:31:39 140338210944960 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
database_1  | 2017-12-14 11:31:39 140338210944960 [Note] InnoDB: Compressed tables use zlib 1.2.8
database_1  | 2017-12-14 11:31:39 140338210944960 [Note] InnoDB: Using Linux native AIO
database_1  | 2017-12-14 11:31:39 140338210944960 [Note] InnoDB: Using SSE crc32 instructions
database_1  | 2017-12-14 11:31:39 140338210944960 [Note] InnoDB: Initializing buffer pool, size = 256.0M
database_1  | 2017-12-14 11:31:39 140338210944960 [Note] InnoDB: Completed initialization of buffer pool
database_1  | 2017-12-14 11:31:39 140338210944960 [Note] InnoDB: Highest supported file format is Barracuda.
database_1  | 2017-12-14 11:31:39 140338210944960 [Note] InnoDB: 128 rollback segment(s) are active.
database_1  | 2017-12-14 11:31:39 140338210944960 [Note] InnoDB: Waiting for purge to start
database_1  | 2017-12-14 11:31:39 140338210944960 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.36-82.2 started; log sequence number 1616707
database_1  | 2017-12-14 11:31:39 140337393694464 [Note] InnoDB: Dumping buffer pool(s) not yet started
database_1  | 
database_1  | PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER !
database_1  | To do so, start the server, then issue the following commands:
database_1  | 
database_1  | '/usr/bin/mysqladmin' -u root password 'new-password'
database_1  | '/usr/bin/mysqladmin' -u root -h  password 'new-password'
database_1  | 
database_1  | Alternatively you can run:
database_1  | '/usr/bin/mysql_secure_installation'
database_1  | 
database_1  | which will also give you the option of removing the test
database_1  | databases and anonymous user created by default.  This is
database_1  | strongly recommended for production servers.
database_1  | 
database_1  | See the MariaDB Knowledgebase at http://mariadb.com/kb or the
database_1  | MySQL manual for more instructions.
database_1  | 
database_1  | Please report any problems at http://mariadb.org/jira
database_1  | 
database_1  | The latest information about MariaDB is available at http://mariadb.org/.
database_1  | You can find additional information about the MySQL part at:
database_1  | http://dev.mysql.com
database_1  | Consider joining MariaDB's strong and vibrant community:
database_1  | https://mariadb.org/get-involved/
database_1  | 
database_1  | Database initialized
database_1  | MySQL init process in progress...
database_1  | 2017-12-14 11:31:42 139939563587520 [Note] mysqld (mysqld 10.1.29-MariaDB-1~jessie) starting as process 154 ...
database_1  | 2017-12-14 11:31:42 139939563587520 [Note] InnoDB: Using mutexes to ref count buffer pool pages
database_1  | 2017-12-14 11:31:42 139939563587520 [Note] InnoDB: The InnoDB memory heap is disabled
database_1  | 2017-12-14 11:31:42 139939563587520 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
database_1  | 2017-12-14 11:31:42 139939563587520 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
database_1  | 2017-12-14 11:31:42 139939563587520 [Note] InnoDB: Compressed tables use zlib 1.2.8
database_1  | 2017-12-14 11:31:42 139939563587520 [Note] InnoDB: Using Linux native AIO
database_1  | 2017-12-14 11:31:42 139939563587520 [Note] InnoDB: Using SSE crc32 instructions
database_1  | 2017-12-14 11:31:42 139939563587520 [Note] InnoDB: Initializing buffer pool, size = 256.0M
database_1  | 2017-12-14 11:31:42 139939563587520 [Note] InnoDB: Completed initialization of buffer pool
database_1  | 2017-12-14 11:31:42 139939563587520 [Note] InnoDB: Highest supported file format is Barracuda.
database_1  | 2017-12-14 11:31:42 139939563587520 [Note] InnoDB: 128 rollback segment(s) are active.
database_1  | 2017-12-14 11:31:42 139939563587520 [Note] InnoDB: Waiting for purge to start
database_1  | 2017-12-14 11:31:42 139939563587520 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.36-82.2 started; log sequence number 1616717
database_1  | 2017-12-14 11:31:42 139938746070784 [Note] InnoDB: Dumping buffer pool(s) not yet started
database_1  | 2017-12-14 11:31:42 139939563587520 [Note] Plugin 'FEEDBACK' is disabled.
database_1  | 2017-12-14 11:31:42 139939563587520 [Warning] 'user' entry 'root@eca312c9edad' ignored in --skip-name-resolve mode.
database_1  | 2017-12-14 11:31:42 139939563587520 [Warning] 'proxies_priv' entry '@% root@eca312c9edad' ignored in --skip-name-resolve mode.
database_1  | 2017-12-14 11:31:42 139939563587520 [Note] mysqld: ready for connections.
database_1  | Version: '10.1.29-MariaDB-1~jessie'  socket: '/var/run/mysqld/mysqld.sock'  port: 0  mariadb.org binary distribution
database_1  | Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
database_1  | 2017-12-14 11:31:43 139939562720000 [Warning] 'proxies_priv' entry '@% root@eca312c9edad' ignored in --skip-name-resolve mode.
database_1  | 2017-12-14 11:31:43 139939562720000 [Warning] 'proxies_priv' entry '@% root@eca312c9edad' ignored in --skip-name-resolve mode.
database_1  | 
database_1  | /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/init_cluster_conf.sh
gelarcluster_database_1 exited with code 2

Am i doing wrong? Can i provide some more output to help you fixing the issue?

a node issue could cause to be locked out from the cluster

Hi,
after sometime in wich the cluster runs great (a bit slow, but works), today i found it down (on the logs, the nodes seems that fail to connect between), so i tryied to investigate the problem, but i can't spend much time to debug while many services was down. All that i know is that yesterday i restarted one node. My question is: the cluster is affected of some known issue? What happen if the container that bootstrap the cluster restarts?

Sorry for my english :D

can't use mode global

Hi,
after some days spent on docker swarm, i'm happy to not have persistent volumes and to see that with many replicas all works good. There is another issue that could cause loosing data on cluster. If i need to boot first 1 replica, and then scale, i can't set global mode in docker swarm (because it start all the replicas contemporary), and with the replicated mode is possible that all the replicas are on the same node, so shutting down the node will delete everything.
The only one solution (not tested), is to enable global mode, but costrain the service to run on node with certain label and initially set it only on one node, and then adding that label to more nodes. The best is to be able to start all the replicas together. Is possible to fix this issue?

Adding healtcheck in docker compose file caused the container to fail

Hi,

I'm running docker 1.13 experimental and use compose file to deploy mariadb in cluster using your image.
The docker compose file looks like:

version: '3'
services:
    mariadb:
      deploy:
        replicas: 1
      image: ${DOCKER_REPOSITORY}openpaas/toughiq/mariadb-cluster:2.0
      networks:
        - default
      volumes:
        - /config/mariadb/maria.cnf:/etc/mysql/conf.d/maria.cnf
      healthcheck:
          test: ["CMD", "ls"]
          interval: 30s
          timeout: 1m30s
          retries: 5
      environment:
        - DB_SERVICE_NAME=mariadb
        - MYSQL_ROOT_PASSWORD=xxxx
        - MYSQL_DATABASE=yyyyy
        - MYSQL_USER=user1
        - MYSQL_PASSWORD=user1

If i put the healthcheck test, the mariadb container failed after 10 sec, and the last line of the log shows:
/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/init_cluster_conf.sh
If I removed the healthcheck test, the mariadb container succeeds running.
BTW, the healthcheck is a dummy check and no matter what i put in the command it causes the container to fail.

Do you have any idea about the reason of the crash ? It works me fine for other container (not mariadb).

Thanks

When reboot master db node then can not join back to the old cluster

Hi,
I run the cluster in 3 db nodes. and initially, the replica is 1 then i scale to 3. So 3 db containers running on 3 physical nodes separately without any issue.
later I reboot docker swarm manager also the node1 master db host. then the db cluster can not join back.
Is anyone have this issue?

Persistence

I realize the description on the readme states that this is supposed to work with no persistent data volumes, but I really like what you've done with this - I just wanted the extra peace of mind of having data persistence. I'd like to see it added as an option.

I made a version that, sensing a volume at /data, manages the mysql datadir in that location. Except, each container gets it's own subfolder. That way you can have multiple containers on each host. Or all the containers can share a network (nfs) location safely. There is also a cleanup function as containers come and go. I'll put in a pull request.