Code Monkey home page Code Monkey logo

wing-binlog-go's Introduction

wing-binlog-go

wing-binlog-go is a mysql(mariadb) master-slave replication client, that uses replication protocol to read events from Mysql or MariaDB's binlog build on top of go-mysql, and send json data via Http, TCP、 Redis queue、kafka or subscribe. It allows you to receive events like insert, update and delete with the original data.

  • wing-binlog-go is a high performance Mysql/MariaDB middleware
  • wing-binlog-go is a light weight Mysql/MariaDB data watch system

wing-binlog-go can run on OSX, Linux and Windows system.

Installation:

  • On Xnix, run ./bin/build.sh
  • On Windows, run bin/build.bat

Project status The project is test with:

  • Mysql 5.5, 5.6 and 5.7
  • MariaDB 5.5, 10.0, 10.1 and 10.2
  • golang version 1.8+
  • consul v1.0.2

Important Mysql server settings

[mysqld]
expire_logs_days         = 10
max_binlog_size          = 100M
server-id		 = 1
log_bin			 = /var/log/mysql/mysql-bin.log
binlog-format            = row

The system variables server_id, log_bin and binlog-format are madantory, and row based replication is required for the project.

Features:

  • TCP protocol support
  • Http protocal support
  • Support client group, broadcast and filters

Use cases:

  • MySQL to NoSQL replication
  • MySQL to search engine replication, such as elasticsearch
  • Realtime analytics
  • Audit

Usage:

  • Help: ./wing-binlog-go -help
  • Start service: ./wing-binlog-go
  • Show version information: ./wing-binlog-go -version
  • Stop service: ./wing-binlog-go -stop
  • Reload service: ./wing-binlog-go -reload all|http|tcp

Known issues:

  1. The underlying package go-mysql does not support compressed binlog
  2. Schema changes after initial binlog position may cause incorrect json package

Special thanks:

Contributors

  • jilieryuyi
  • mia0x75

Help:

QQ group: 535218312 document

wing-binlog-go's People

Contributors

jilieryuyi avatar mia0x75 avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar

Forkers

zhanglei mia0x75 jackerzhou isgasho jangocheng yangyangyang

wing-binlog-go's Issues

正常关闭的信息提示

用-stop关闭

INFO[2017-12-15] closing canal                                
INFO[2017-12-15] syncer is closing...                         
WARN[2017-12-15] tcp服务发生错误：accept tcp [::]:9998: use of closed network connection 
WARN[2017-12-15] tcp服务发生错误：accept tcp [::]:9998: use of closed network connection 
WARN[2017-12-15] tcp服务发生错误：accept tcp [::]:9998: use of closed network connection 
ERRO[2017-12-15] canal start sync binlog err: context canceled 
FATA[2017-12-15] binlog服务：start canal err context canceled

这个是不是不应该有WARN/ERRO/FATA类的错误信息？

乱码问题

INFO[2017-12-04 11:45:42] create BinlogSyncer with config {3017243 mariadb 172.30.17.250 3306 replicate gbk false false false 0 30s 0s}
INFO[2017-12-04 11:45:42] websocket服务-等待新的连接...
INFO[2017-12-04 11:45:42] binlog调试： mysql-bin.000072 221909292
INFO[2017-12-04 11:45:42] websocket服务-监听: 0.0.0.0:9997
INFO[2017-12-04 11:45:42] skip dump, use last binlog replication pos (mysql-bin.000072, 221909292) or GTID %!s()
[martini] listening on 0.0.0.0:9997 (development)
INFO[2017-12-04 11:45:42] begin to sync binlog from position (mysql-bin.000072, 221909292)
INFO[2017-12-04 11:45:42] register slave for master server a.b.c.d:3306
INFO[2017-12-04 11:45:42] tcp服务-等待新的连接...
INFO[2017-12-04 11:45:42] start sync binlog at binlog file (mysql-bin.000072, 221909292)
INFO[2017-12-04 11:45:42] rotate to (mysql-bin.000072, 221909292)
INFO[2017-12-04 11:45:42] rotate binlog to (mysql-bin.000072, 221909292)
INFO[2017-12-04 11:45:42] binlog发送广播： %lms.tb_riskmanage_idempotence_control{"database":"lms"uuid":"bb81abc4eef3452281df1e490a825e33","timestamp":"2017-11-27 10:44:44"},"event_type":"insert","time":1512359142},"event_index":5821,"table":"tb_riskmanage_idempotence_control"}
INFO[2017-12-04 11:45:42] http服务-发送广播
INFO[2017-12-04 11:45:42] kafka服务-发送消息
INFO[2017-12-04 11:45:42] 37 -- [108 109 115 46 116 98 95 114 105 115 107 109 97 110 97 103 101 95 105 100 101 109 112 111 116 101 110 99 101 95 99 111 110 116 114 111 108] -- [123 34 100 97 116 97 98 97 115 101 34 58 34 108 109 115 34 117 117 105 100 34 58 34 98 98 56 49 97 98 99 52 101 101 102 51 52 53 50 50 56 49 100 102 49 101 52 57 48 97 56 50 53 101 51 51 34 44 34 116 105 109 101 115 116 97 109 112 34 58 34 50 48 49 55 45 49 49 45 50 55 32 49 48 58 52 52 58 52 52 34 125 44 34 101 118 101 110 116 95 116 121 112 101 34 58 34 105 110 115 101 114 116 34 44 34 116 105 109 101 34 58 49 53 49 50 51 53 57 49 52 50 125 44 34 101 118 101 110 116 95 105 110 100 101 120 34 58 53 56 50 49 44 34 116 97 98 108 101 34 58 34 116 98 95 114 105 115 107 109 97 110 97 103 101 95 105 100 101 109 112 111 116 101 110 99 101 95 99 111 110 116 114 111 108 34 125]
INFO[2017-12-04 11:45:42] binlog发送广播： decision.tb_lb_decision_grade{"database":"decision"id":22755401,"apply_code":"00511711278249","dimension":"稳定性","seq":9,"name":"本地居住年限","value":"��60个月","score":39,"last_update_time":"2017-11-27 10:44:44"},"event_type":"insert","time":1512359142},"event_index":5852,"table":"tb_lb_decision_grade"}
INFO[2017-12-04 11:45:42] tcp服务-广播
INFO[2017-12-04 11:45:42] tcp服务-没有连接的客户端
INFO[2017-12-04 11:45:42] http服务-发送广播
INFO[2017-12-04 11:45:42] kafka服务-发送消息
INFO[2017-12-04 11:45:42] binlog发送广播： decision.tb_lb_decision_grade{"database":"decision"id":22755414,"apply_code":"00511711278249","dimension":"车辆特征","seq":22,"name":"评估价","value":"＞5万，≤10万","score":12,"last_update_time":"2017-11-27 10:44:44"},"event_type":"insert","time":1512359142},"event_index":5865,"table":"tb_lb_decision_grade"}
INFO[2017-12-04 11:45:42] http服务-发送广播
INFO[2017-12-04 11:45:42] kafka服务-发送消息
INFO[2017-12-04 11:45:42] binlog发送广播： ++.+b__y__+-g{"da+aba_e":"++"ID":11647087,"USERNAME":"+i++i1","OPDATE":"2017-11-27 10:45:30","MODULECODE":"特批业务","OPCONTENT":"提前结清申请","OPSIGN":"保存成功!","OPSQL":"","OPIP":"112.103.18.174","HOSTMAC":""},"e+e+++y-e":"i+e+","+i+e":1512359142},"e+e++i+de|":5866,"+ab+e":"+b__y_+-g"}
INFO[2017-12-04 11:45:42] +c-服务-广播
INFO[2017-12-04 11:45:42] +c-服务-没有连接的客户端
INFO[2017-12-04 11:45:42] h++-服务-发送广播
INFO[2017-12-04 11:45:42] +af+a服务-发送消息
INFO[2017-12-04 11:45:42] bi++-g发送广播： #++.+b_c-++ac+ide+--+e+ce_c-++-+{"da+aba_e":"++"++id":"220eaa4b-981a-4092-aa8d-e198cacbc921","+i+e+a+-":"2017-11-27 10:44:45"},"e+e++_+y-e":"i+e+","+i+e":1512359142},"e+e++_i+de|":5867,"+ab+e":"+b_c-++_ac+ide+--+e+ce_c-++-+"}
INFO[2017-12-04 11:45:42] +c-服务-广播
ESC[36mINFOESC[0m[2017-12-04 11:56:00] binlog发送广播： ^]^@decision.tb_lb_decision_grade{"database":"decision"id":24464933,"apply_code":"2941171204238","dimension":"稳定性","seq":9,"name":"本
地居住年限","value":"<U+FEFF>＞60个月","score":39,"last_update_time":"2017-12-04 11:55:07"},"event_type":"delete","time":1512359760},"event_index":739,"table":"tb_lb_decision_grade"}

SQL类型支持，做了一个测试用例

建表：

--所有无符号的定点和浮点数不做测试
CREATE TABLE dt_fixed_1(c FIXED(38,0));
CREATE TABLE dt_fixed_2(c FIXED(38,19));
CREATE TABLE dt_fixed_3(c FIXED(38,30));

CREATE TABLE dt_real_1(c REAL(38,0));
CREATE TABLE dt_real_2(c REAL(38,19));
CREATE TABLE dt_real_3(c REAL(38,30));

CREATE TABLE dt_decimal_1(c DECIMAL(38,0));
CREATE TABLE dt_decimal_2(c DECIMAL(38,19));
CREATE TABLE dt_decimal_3(c DECIMAL(38,38));

CREATE TABLE dt_dec_1(c DEC(38,0));
CREATE TABLE dt_dec_2(c DEC(38,19));
CREATE TABLE dt_dec_3(c DEC(38,38));

CREATE TABLE dt_numeric_1(c NUMERIC(38,0));
CREATE TABLE dt_numeric_2(c NUMERIC(38,19));
CREATE TABLE dt_numeric_3(c NUMERIC(38,38));

CREATE TABLE dt_float_1(c FLOAT(38,0));
CREATE TABLE dt_float_2(c FLOAT(38,19));
CREATE TABLE dt_float_3(c FLOAT(38,30));

CREATE TABLE dt_doubld_1(c DOUBLE(38,0));
CREATE TABLE dt_doubld_2(c DOUBLE(38,19));
CREATE TABLE dt_doubld_3(c DOUBLE(38,30));

CREATE TABLE dt_bit (c BIT);
CREATE TABLE dt_bool (c BOOL);
CREATE TABLE dt_boolean (c BOOLEAN);

CREATE TABLE dt_int8 (c TINYINT);
CREATE TABLE dt_uint8 (c TINYINT UNSIGNED);
CREATE TABLE dt_int16 (c SMALLINT);
CREATE TABLE dt_uint16 (c SMALLINT UNSIGNED);
CREATE TABLE dt_int24 (c MEDIUMINT);
CREATE TABLE dt_uint24 (c MEDIUMINT UNSIGNED);
CREATE TABLE dt_int32 (c INT);
CREATE TABLE dt_uint32 (c INT UNSIGNED);
CREATE TABLE dt_int64 (c BIGINT);
CREATE TABLE dt_uint64 (c BIGINT UNSIGNED);

CREATE TABLE dt_binary (c BINARY(16));
CREATE TABLE dt_varbinary (c VARBINARY(16));

CREATE TABLE dt_tinyblob (c TINYBLOB);
CREATE TABLE dt_mediumblob (c MEDIUMBLOB);
CREATE TABLE dt_blob (c BLOB);
CREATE TABLE dt_longblob (c LONGBLOB);

CREATE TABLE dt_tinytext (c TINYTEXT);
CREATE TABLE dt_mediumtext (c MEDIUMTEXT);
CREATE TABLE dt_text (c TEXT);
CREATE TABLE dt_longtext (c LONGTEXT);

CREATE TABLE dt_date (c DATE);
CREATE TABLE dt_time (c TIME);
CREATE TABLE dt_datetime (c DATETIME);
CREATE TABLE dt_timestamp (c TIMESTAMP);
CREATE TABLE dt_year_1 (c YEAR(2)); --5.5后不再支持，不测试
CREATE TABLE dt_year_2 (c YEAR(4));

CREATE TABLE dt_char (c CHAR(32));
CREATE TABLE dt_varchar (c VARCHAR(32));

CREATE TABLE dt_enum (c ENUM('F', 'M'));
CREATE TABLE dt_set (c SET ('Length', 'Height', 'Width', 'Weight', 'Color'));

数据：

insert into dt_date VALUES ('1000-01-01');
insert into dt_date VALUES ('9999-12-31');
insert into dt_date VALUES ('2017-12-06');

insert into dt_time VALUES ('-838:59:59.999999'); --丢失精度
insert into dt_time VALUES ('838:59:59.999999');  --丢失精度
insert into dt_time VALUES ('90:00:00');
insert into dt_time VALUES ('800:00:00');
insert into dt_time VALUES (800);
insert into dt_time VALUES (22);
insert into dt_time VALUES (151413);
insert into dt_time VALUES ('9:6:3');
insert into dt_time VALUES ('12 09');

insert into dt_timestamp VALUES (CURRENT_TIMESTAMP());
insert into dt_timestamp VALUES (CURRENT_TIMESTAMP());
insert into dt_timestamp VALUES (CURRENT_TIMESTAMP());
insert into dt_timestamp VALUES (CURRENT_TIMESTAMP());
insert into dt_timestamp VALUES (CURRENT_TIMESTAMP());

insert into dt_datetime VALUES ('1000-01-01 00:00:00.000000'); --丢失精度
insert into dt_datetime VALUES ('9999-12-31 23:59:59.999999'); --丢失精度
insert into dt_datetime VALUES (NOW());

insert into dt_tinytext VALUES ('');
insert into dt_tinytext VALUES (REPEAT('0', 255)); --超过252就不能处理
insert into dt_tinytext VALUES ('1234567890');
insert into dt_tinytext VALUES ('1234567890~!@#$%^&*()_+":?><`'); --引号没处理
insert into dt_tinytext VALUES ('abcdefghijklmnopqrstuvwxyz');
insert into dt_tinytext VALUES ('ABCDEFGHIJKLMNOPQRSTUVWXYZ');
insert into dt_tinytext VALUES ('你好');
insert into dt_tinytext VALUES ('おはようございます');
insert into dt_tinytext VALUES ('привет');
insert into dt_tinytext VALUES ('namasté');
insert into dt_tinytext VALUES ('여보세요');

insert into dt_text VALUES ('');
insert into dt_text VALUES (REPEAT('0', 65535));--超过252就不能处理
insert into dt_text VALUES (REPEAT('0', 32767));--超过252就不能处理

--应该跟tinytext有相同问题
insert into dt_tinyblob VALUES (0x00010203040506070809);
insert into dt_tinyblob VALUES (0x000102030405060708090A0B0C0D0E0F101112131415161718191A1B1C1D1E1F202122232425262728292A2B2C2D2E2F303132333435363738393A3B3C3D3E3F404142434445464748494A4B4C4D4E4F505152535455565758595A5B5C5D5E5F606162636465666768696A6B6C6D6E6F707172737475767778797A7B7C7D7E7F808182838485868788898A8B8C8D8E8F909192939495969798999A9B9C9D9E9FA0A1A2A3A4A5A6A7A8A9AAABACADAEAFB0B1B2B3B4B5B6B7B8B9BABBBCBDBEBFC0C1C2C3C4C5C6C7C8C9CACBCCCDCECFD0D1D2D3D4D5D6D7D8D9DADBDCDDDEDFE0E1E2E3E4E5E6E7E8E9EAEBECEDEEEFF0F1F2F3F4F5F6F7F8F9FAFBFCFDFE);

insert into dt_bit VALUES (0);
insert into dt_bit VALUES (1);
insert into dt_bool VALUES (0);
insert into dt_bool VALUES (1);
insert into dt_boolean VALUES (0);
insert into dt_boolean VALUES (1);

insert into dt_year_2 VALUES (2000);                                         --一般值
insert into dt_year_2 VALUES (2155);                                         --上边界
insert into dt_year_2 VALUES (0000);                                         --下边界

insert into dt_int8 VALUES (-128);                                         --下边界
insert into dt_int8 VALUES (127);                                          --上边界
insert into dt_int8 VALUES (FLOOR((RAND()-0.5)*255));                      --一般值
insert into dt_int8 VALUES (FLOOR((RAND()-0.5)*255));                      --一般值
insert into dt_int8 VALUES (FLOOR((RAND()-0.5)*255));                      --一般值
insert into dt_int8 VALUES (FLOOR((RAND()-0.5)*255));                      --一般值
insert into dt_int8 VALUES (FLOOR((RAND()-0.5)*255));                      --一般值

insert into dt_uint8 VALUES (0);                                           --下边界
insert into dt_uint8 VALUES (255);                                         --上边界
insert into dt_uint8 VALUES (FLOOR(RAND()*255));                           --一般值
insert into dt_uint8 VALUES (FLOOR(RAND()*255));                           --一般值
insert into dt_uint8 VALUES (FLOOR(RAND()*255));                           --一般值
insert into dt_uint8 VALUES (FLOOR(RAND()*255));                           --一般值
insert into dt_uint8 VALUES (FLOOR(RAND()*255));                           --一般值

insert into dt_int16 VALUES (-32768);                                      --下边界
insert into dt_int16 VALUES (32767);                                       --上边界
insert into dt_int16 VALUES (FLOOR((RAND()-0.5)*65535));                   --一般值
insert into dt_int16 VALUES (FLOOR((RAND()-0.5)*65535));                   --一般值
insert into dt_int16 VALUES (FLOOR((RAND()-0.5)*65535));                   --一般值
insert into dt_int16 VALUES (FLOOR((RAND()-0.5)*65535));                   --一般值
insert into dt_int16 VALUES (FLOOR((RAND()-0.5)*65535));                   --一般值

insert into dt_uint16 VALUES (0);                                          --下边界
insert into dt_uint16 VALUES (65535);                                      --上边界
insert into dt_uint16 VALUES (FLOOR(RAND()*65535));                        --一般值
insert into dt_uint16 VALUES (FLOOR(RAND()*65535));                        --一般值
insert into dt_uint16 VALUES (FLOOR(RAND()*65535));                        --一般值
insert into dt_uint16 VALUES (FLOOR(RAND()*65535));                        --一般值
insert into dt_uint16 VALUES (FLOOR(RAND()*65535));                        --一般值

insert into dt_int24 VALUES (-8388608);                                    --下边界
insert into dt_int24 VALUES (8388607);                                     --上边界
insert into dt_int24 VALUES (FLOOR((RAND()-0.5)*16777215));                --一般值
insert into dt_int24 VALUES (FLOOR((RAND()-0.5)*16777215));                --一般值
insert into dt_int24 VALUES (FLOOR((RAND()-0.5)*16777215));                --一般值
insert into dt_int24 VALUES (FLOOR((RAND()-0.5)*16777215));                --一般值
insert into dt_int24 VALUES (FLOOR((RAND()-0.5)*16777215));                --一般值

insert into dt_uint24 VALUES (0);                                          --下边界
insert into dt_uint24 VALUES (16777215);                                   --上边界
insert into dt_uint24 VALUES (FLOOR(RAND()*16777215));                     --一般值
insert into dt_uint24 VALUES (FLOOR(RAND()*16777215));                     --一般值
insert into dt_uint24 VALUES (FLOOR(RAND()*16777215));                     --一般值
insert into dt_uint24 VALUES (FLOOR(RAND()*16777215));                     --一般值
insert into dt_uint24 VALUES (FLOOR(RAND()*16777215));                     --一般值

insert into dt_int32 VALUES (-2147483648);                                 --下边界
insert into dt_int32 VALUES (2147483647);                                  --上边界
insert into dt_int32 VALUES (FLOOR((RAND()-0.5)*4294967295));              --一般值
insert into dt_int32 VALUES (FLOOR((RAND()-0.5)*4294967295));              --一般值
insert into dt_int32 VALUES (FLOOR((RAND()-0.5)*4294967295));              --一般值
insert into dt_int32 VALUES (FLOOR((RAND()-0.5)*4294967295));              --一般值
insert into dt_int32 VALUES (FLOOR((RAND()-0.5)*4294967295));              --一般值

insert into dt_uint32 VALUES (0);                                          --下边界
insert into dt_uint32 VALUES (4294967295);                                 --上边界
insert into dt_uint32 VALUES (FLOOR(RAND()*4294967295));                   --一般值
insert into dt_uint32 VALUES (FLOOR(RAND()*4294967295));                   --一般值
insert into dt_uint32 VALUES (FLOOR(RAND()*4294967295));                   --一般值
insert into dt_uint32 VALUES (FLOOR(RAND()*4294967295));                   --一般值
insert into dt_uint32 VALUES (FLOOR(RAND()*4294967295));                   --一般值

insert into dt_int64 VALUES (-9223372036854775808);                        --下边界
insert into dt_int64 VALUES (9223372036854775807);                         --上边界
insert into dt_int64 VALUES (FLOOR((RAND()-0.5)*18446744073709551615));    --一般值
insert into dt_int64 VALUES (FLOOR((RAND()-0.5)*18446744073709551615));    --一般值
insert into dt_int64 VALUES (FLOOR((RAND()-0.5)*18446744073709551615));    --一般值
insert into dt_int64 VALUES (FLOOR((RAND()-0.5)*18446744073709551615));    --一般值
insert into dt_int64 VALUES (FLOOR((RAND()-0.5)*18446744073709551615));    --一般值

insert into dt_uint64 VALUES (0);                                          --下边界
insert into dt_uint64 VALUES (18446744073709551615);                       --上边界
insert into dt_uint64 VALUES (FLOOR(RAND()*18446744073709551615));         --一般值
insert into dt_uint64 VALUES (FLOOR(RAND()*18446744073709551615));         --一般值
insert into dt_uint64 VALUES (FLOOR(RAND()*18446744073709551615));         --一般值
insert into dt_uint64 VALUES (FLOOR(RAND()*18446744073709551615));         --一般值
insert into dt_uint64 VALUES (FLOOR(RAND()*18446744073709551615));         --一般值

INSERT INTO dt_doubld_1 VALUES (+99999999999999999999999999999999999999);  --上边界
INSERT INTO dt_doubld_1 VALUES (-99999999999999999999999999999999999999);  --下边界
INSERT INTO dt_doubld_1 VALUES (+12345678912345678912345678912345678912);  --一般值
INSERT INTO dt_doubld_1 VALUES (-12345678912345678912345678912345678912);  --一般值

INSERT INTO dt_doubld_2 VALUES (+9999999999999999999.9999999999999999999); --上边界
INSERT INTO dt_doubld_2 VALUES (-9999999999999999999.9999999999999999999); --下边界
INSERT INTO dt_doubld_2 VALUES (+1234567891234567891.1234567891234567891); --一般值
INSERT INTO dt_doubld_2 VALUES (-1234567891234567891.1234567891234567891); --一般值

INSERT INTO dt_doubld_3 VALUES (+0.99999999999999999999999999999999999999);--上边界
INSERT INTO dt_doubld_3 VALUES (-0.99999999999999999999999999999999999999);--下边界
INSERT INTO dt_doubld_3 VALUES (+0.12345678912345678912345678912345678912);--一般值
INSERT INTO dt_doubld_3 VALUES (-0.12345678912345678912345678912345678912);--一般值

INSERT INTO dt_float_1 VALUES (+99999999999999999999999999999999999999);   --上边界
INSERT INTO dt_float_1 VALUES (-99999999999999999999999999999999999999);   --下边界
INSERT INTO dt_float_1 VALUES (+12345678912345678912345678912345678912);   --一般值
INSERT INTO dt_float_1 VALUES (-12345678912345678912345678912345678912);   --一般值

INSERT INTO dt_float_2 VALUES (+9999999999999999999.9999999999999999999);  --上边界
INSERT INTO dt_float_2 VALUES (-9999999999999999999.9999999999999999999);  --下边界
INSERT INTO dt_float_2 VALUES (+1234567891234567891.1234567891234567891);  --一般值
INSERT INTO dt_float_2 VALUES (-1234567891234567891.1234567891234567891);  --一般值

INSERT INTO dt_float_3 VALUES (+99999999.999999999999999999999999999999); --上边界
INSERT INTO dt_float_3 VALUES (-99999999.999999999999999999999999999999); --下边界
INSERT INTO dt_float_3 VALUES (+12345678.123456789123456789123456789123); --一般值
INSERT INTO dt_float_3 VALUES (-12345678.123456789123456789123456789123); --一般值

INSERT INTO dt_numeric_1 VALUES (+99999999999999999999999999999999999999);  --上边界
INSERT INTO dt_numeric_1 VALUES (-99999999999999999999999999999999999999);  --下边界
INSERT INTO dt_numeric_1 VALUES (+12345678912345678912345678912345678912);  --一般值
INSERT INTO dt_numeric_1 VALUES (-12345678912345678912345678912345678912);  --一般值

INSERT INTO dt_numeric_2 VALUES (+9999999999999999999.9999999999999999999); --上边界
INSERT INTO dt_numeric_2 VALUES (-9999999999999999999.9999999999999999999); --下边界
INSERT INTO dt_numeric_2 VALUES (+1234567891234567891.1234567891234567891); --一般值
INSERT INTO dt_numeric_2 VALUES (-1234567891234567891.1234567891234567891); --一般值

INSERT INTO dt_numeric_3 VALUES (+0.99999999999999999999999999999999999999);--上边界
INSERT INTO dt_numeric_3 VALUES (-0.99999999999999999999999999999999999999);--下边界
INSERT INTO dt_numeric_3 VALUES (+0.12345678912345678912345678912345678912);--一般值
INSERT INTO dt_numeric_3 VALUES (-0.12345678912345678912345678912345678912);--一般值

INSERT INTO dt_dec_1 VALUES (+99999999999999999999999999999999999999);  --上边界
INSERT INTO dt_dec_1 VALUES (-99999999999999999999999999999999999999);  --下边界
INSERT INTO dt_dec_1 VALUES (+12345678912345678912345678912345678912);  --一般值
INSERT INTO dt_dec_1 VALUES (-12345678912345678912345678912345678912);  --一般值

INSERT INTO dt_dec_2 VALUES (+9999999999999999999.9999999999999999999); --上边界
INSERT INTO dt_dec_2 VALUES (-9999999999999999999.9999999999999999999); --下边界
INSERT INTO dt_dec_2 VALUES (+1234567891234567891.1234567891234567891); --一般值
INSERT INTO dt_dec_2 VALUES (-1234567891234567891.1234567891234567891); --一般值

INSERT INTO dt_dec_3 VALUES (+0.99999999999999999999999999999999999999);--上边界
INSERT INTO dt_dec_3 VALUES (-0.99999999999999999999999999999999999999);--下边界
INSERT INTO dt_dec_3 VALUES (+0.12345678912345678912345678912345678912);--一般值
INSERT INTO dt_dec_3 VALUES (-0.12345678912345678912345678912345678912);--一般值

INSERT INTO dt_decimal_1 VALUES (+99999999999999999999999999999999999999);  --上边界
INSERT INTO dt_decimal_1 VALUES (-99999999999999999999999999999999999999);  --下边界
INSERT INTO dt_decimal_1 VALUES (+12345678912345678912345678912345678912);  --一般值
INSERT INTO dt_decimal_1 VALUES (-12345678912345678912345678912345678912);  --一般值

INSERT INTO dt_decimal_2 VALUES (+9999999999999999999.9999999999999999999); --上边界
INSERT INTO dt_decimal_2 VALUES (-9999999999999999999.9999999999999999999); --下边界
INSERT INTO dt_decimal_2 VALUES (+1234567891234567891.1234567891234567891); --一般值
INSERT INTO dt_decimal_2 VALUES (-1234567891234567891.1234567891234567891); --一般值

INSERT INTO dt_decimal_3 VALUES (+0.99999999999999999999999999999999999999);--上边界
INSERT INTO dt_decimal_3 VALUES (-0.99999999999999999999999999999999999999);--下边界
INSERT INTO dt_decimal_3 VALUES (+0.12345678912345678912345678912345678912);--一般值
INSERT INTO dt_decimal_3 VALUES (-0.12345678912345678912345678912345678912);--一般值

应该跟以下类型有关（精度问题）：

   dt_decimal
   dt_dec
   dt_numeric
   dt_fixed
   dt_float
   dt_double
   dt_real

目前问题

  • datetime 精度问题
  • time 精度问题
  • 大于252长度问题 text/blob/varchar/varbinary等
  • 定点数/浮点数问题

关于后台进程问题

建议有 -d或者--daemon选项，toml配置日志输入位置，有rotate更好。然后直接启动到后台，日志输出到文件。

build.sh脚本错误

cp -rf ${current_path}/src/config/* ${current_path}/bin/config/

=>

cp -rf ${current_path}/src/config/ ${current_path}/bin/config/

数组下标越界src/library/binlog.go

func (h *binlogHandler) OnRow(e *canal.RowsEvent) error

原因：
开始的日志点t1，在服务器启动前，数据库对表foo做了DDL操作，这样服务器启动后获取到的row结构跟日志开始处t1已经不相同。

处理QUIT

kill -QUIT xxx

INFO[2017-12-13 23:38:34] closing canal                                
INFO[2017-12-13 23:38:34] syncer is closing...                         
ERRO[2017-12-13 23:38:34] canal start sync binlog err: context canceled 
FATA[2017-12-13 23:38:34] Binlog::Start - binlog服务：start canal err context canceled

这个应该收到信号，正常关闭吧？打ERRO/FATA是什么原因？

端口被占用

Step 1, 运行服务，用nohup送到后台
Step 2, 运行 --version，提示端口被占用
INFO[2017-12-08 14:28:06] listen tcp 0.0.0.0:6060: bind: address already in use

看来init先于main执行。

INT(3)类型，报binlog未知的行

&{Name:interview_process Type:1 Collation: RawType:int(3) IsAuto:false IsUnsigned:false EnumValues:[] SetValues:[]}

也有可能根schema变更有关，我再确认。

MariaDB 10.2.x 数据长度截断问题

repo:

cat > /etc/yum.repos.d/MariaDB.repo <<EOF
[MariaDB]
name     = MariaDB
baseurl  = http://ftp.kaist.ac.kr/mariadb/yum/10.2/centos7-amd64
gpgkey   = http://yum.mariadb.org/RPM-GPG-KEY-MariaDB
gpgcheck = 1

;baseurl  = http://yum.mariadb.org/10.2/centos7-amd64
EOF

安装：

yum install MariaDB-client MariaDB-common MariaDB-compat MariaDB-server MariaDB-shared -y

测试：

create table ut_truncate (c TEXT);
insert into ut_truncate VALUES (REPEAT('0', 252));
insert into ut_truncate VALUES (REPEAT('1', 253));
insert into ut_truncate VALUES (REPEAT('2', 254));
insert into ut_truncate VALUES (REPEAT('3', 255));
insert into ut_truncate VALUES (REPEAT('4', 32767));

现象，长度大于252时，程序没有任何反应。就好像根本没有这个日志一样。

Websocket会有重复消息问题

  1. 服务端启动
  2. 浏览器端启动
  3. 连接成功后，关闭服务器，模拟服务崩溃
  4. 服务器重新上线
  5. 数据库操作

会看到重复推送消息到websocket，Http和Tcp好像正常。

client.js
2017-12-02 08:24:00 心跳返回值： ok
client.js:42:2
2017-12-02 08:24:02 事件返回值： 1 {"database":"drafts","event":{"data":{"id":9,"value":"Allen"},"event_type":"insert","time":1512174242},"event_index":6,"table":"bar"}
client.js:42:2
2017-12-02 08:24:02 事件返回值： 2 {"database":"drafts","event":{"data":{"id":9,"value":"Allen"},"event_type":"insert","time":1512174242},"event_index":6,"table":"bar"}
client.js:42:2
2017-12-02 08:24:02 事件返回值： 3 {"database":"drafts","event":{"data":{"id":9,"value":"Allen"},"event_type":"insert","time":1512174242},"event_index":6,"table":"bar"}
client.js:42:2
2017-12-02 08:24:02 事件返回值： 4 {"database":"drafts","event":{"data":{"id":9,"value":"Allen"},"event_type":"insert","time":1512174242},"event_index":6,"table":"bar"}
client.js:42:2
2017-12-02 08:24:02 事件返回值： 5 {"database":"drafts","event":{"data":{"id":9,"value":"Allen"},"event_type":"insert","time":1512174242},"event_index":6,"table":"bar"}
client.js:42:2
2017-12-02 08:24:02 事件返回值： 6 {"database":"drafts","event":{"data":{"id":9,"value":"Allen"},"event_type":"insert","time":1512174242},"event_index":6,"table":"bar"}
client.js:42:2
2017-12-02 08:24:02 事件返回值： 7 {"database":"drafts","event":{"data":{"id":9,"value":"Allen"},"event_type":"insert","time":1512174242},"event_index":6,"table":"bar"}
client.js:42:2
2017-12-02 08:24:02 事件返回值： 8 {"database":"drafts","event":{"data":{"id":9,"value":"Allen"},"event_type":"insert","time":1512174242},"event_index":6,"table":"bar"}
client.js:42:2
2017-12-02 08:24:02 事件返回值： 9 {"database":"drafts","event":{"data":{"id":9,"value":"Allen"},"event_type":"insert","time":1512174242},"event_index":6,"table":"bar"}
client.js:42:2
2017-12-02 08:24:02 事件返回值： 10 {"database":"drafts","event":{"data":{"id":9,"value":"Allen"},"event_type":"insert","time":1512174242},"event_index":6,"table":"bar"}
client.js:42:2
2017-12-02 08:24:02 事件返回值： 11 {"database":"drafts","event":{"data":{"id":9,"value":"Allen"},"event_type":"insert","time":1512174242},"event_index":6,"table":"bar"}
client.js:42:2
2017-12-02 08:24:02 事件返回值： 12 {"database":"drafts","event":{"data":{"id":9,"value":"Allen"},"event_type":"insert","time":1512174242},"event_index":6,"table":"bar"}
client.js:42:2
2017-12-02 08:24:02 事件返回值： 13 {"database":"drafts","event":{"data":{"id":9,"value":"Allen"},"event_type":"insert","time":1512174242},"event_index":6,"table":"bar"}
client.js:42:2
2017-12-02 08:24:02 事件返回值： 14 {"database":"drafts","event":{"data":{"id":9,"value":"Allen"},"event_type":"insert","time":1512174242},"event_index":6,"table":"bar"}

HTTP方式接收数据造成大量TIME_WAIT连接 

TIME-WAIT   0      0 $SERVER:34874 $CLIENT:8125
TIME-WAIT   0      0 $SERVER:29442 $CLIENT:8125
TIME-WAIT   0      0 $SERVER:31806 $CLIENT:8125
TIME-WAIT   0      0 $SERVER:36626 $CLIENT:8125
TIME-WAIT   0      0 $SERVER:30914 $CLIENT:8125
TIME-WAIT   0      0 $SERVER:33906 $CLIENT:8125
TIME-WAIT   0      0 $SERVER:30114 $CLIENT:8125

这种权限问题能处理么？

用root启动
用普通用户stop

$ bin/wing-binlog-go -stop
PANI[2017-12-16 20:25:15] unix服务客户端异常：dial unix .../wing-binlog-go.sock: connect: permission denied 
panic: (*logrus.Entry) (0x871a80,0xc420070960)
.
.
.

JSON编码问题

字段类型：
name varchar(100) CHARACTER SET gbk DEFAULT NULL,

输出：
"name":"\ufffd\ufffd11\ufffd\ufffd\ufffd\ufffd\ufffd\ufffd\ufffd佱\ufffd\ufffd\ufffd\ufffd\ufffd\ufffdŬ\ufffd\ufffd\ufffd\ufffd\ufffdҶ\ufffd֪\ufffd\ufffd\ufffd\ufffd"

真实值：
【11月社区颁奖】你的努力，我都知道！

Step 1:
CREATE TABLE ut_gbk (
name varchar(100) CHARACTER SET gbk DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4

Step 2:
insert into ut_gbk values ('【11月社区颁奖】你的努力，我都知道！');

程序崩溃

panic: runtime error: slice bounds out of range

goroutine 41 [running]:
library/binlog.(*binlogHandler).getPoint(0xc42006e1f8, 0xc42703520d, 0x6, 0x3, 0x3, 0xc420171000)
/home/mxu/Projects/src/github.com/jilieryuyi/wing-binlog-go/src/library/binlog/handler.go:35 +0x13f
library/binlog.(*binlogHandler).append(0xc42006e1f8, 0xc423ff3a18, 0x90b3a0, 0xc4242c8318, 0xc42703d6c0)
/home/mxu/Projects/src/github.com/jilieryuyi/wing-binlog-go/src/library/binlog/handler.go:115 +0xa65
library/binlog.(*binlogHandler).OnRow(0xc42006e1f8, 0xc42656ba10, 0x3, 0xc426f7df80)
/home/mxu/Projects/src/github.com/jilieryuyi/wing-binlog-go/src/library/binlog/handler.go:229 +0x22ee
github.com/siddontang/go-mysql/canal.(*Canal).handleRowsEvent(0xc420096dd0, 0xc423f92ff0, 0x10, 0xc407bad2b3)
/home/mxu/Projects/src/github.com/jilieryuyi/wing-binlog-go/vendor/src/github.com/siddontang/go-mysql/canal/sync.go:183 +0x216
github.com/siddontang/go-mysql/canal.(*Canal).runSyncBinlog(0xc420096dd0, 0x0, 0x0)
/home/mxu/Projects/src/github.com/jilieryuyi/wing-binlog-go/vendor/src/github.com/siddontang/go-mysql/canal/sync.go:81 +0x898
github.com/siddontang/go-mysql/canal.(*Canal).run(0xc420096dd0, 0x0, 0x0)
/home/mxu/Projects/src/github.com/jilieryuyi/wing-binlog-go/vendor/src/github.com/siddontang/go-mysql/canal/canal.go:201 +0x151
github.com/siddontang/go-mysql/canal.(*Canal).Run(0xc420096dd0, 0xc420012a00, 0x10)
/home/mxu/Projects/src/github.com/jilieryuyi/wing-binlog-go/vendor/src/github.com/siddontang/go-mysql/canal/canal.go:170 +0x2b
github.com/siddontang/go-mysql/canal.(*Canal).RunFrom(0xc420096dd0, 0xc420012a00, 0x10, 0x7b89bd9, 0x0, 0x0)
/home/mxu/Projects/src/github.com/jilieryuyi/wing-binlog-go/vendor/src/github.com/siddontang/go-mysql/canal/canal.go:178 +0x5d
library/binlog.(*Binlog).Start.func1(0xc42006e1e0)
/home/mxu/Projects/src/github.com/jilieryuyi/wing-binlog-go/src/library/binlog/binlog.go:111 +0x54
created by library/binlog.(*Binlog).Start
/home/mxu/Projects/src/github.com/jilieryuyi/wing-binlog-go/src/library/binlog/binlog.go:105 +0x1e5

systemctl服务 

[Unit]
Description=Binlog
After=syslog.target
After=network.target

[Service]
#LimitMEMLOCK=infinity
#LimitNOFILE=65535
Type=simple
User=wbl
Group=wbl
ExecStart=/usr/local/wbl/wing-binlog-go
Restart=always

[Install]
WantedBy=multi-user.target

systemctl start wbl ## 这个可以

systemctl stop wbl

# systemctl status wbl.service
● wbl.service - Binlog
   Loaded: loaded (/usr/lib/systemd/system/wbl.service; disabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Thu 2017-12-14 13:25:08 CST; 1s ago
  Process: 2042 ExecStart=/usr/local/wbl/wing-binlog-go (code=exited, status=1/FAILURE)
 Main PID: 2042 (code=exited, status=1/FAILURE)

Dec 14 13:25:08 rx-wj39106 systemd[1]: Stopping wbl...
Dec 14 13:25:08 rx-wj39106 wbl[2042]: INFO[2017-12-14 13:25:08] closing canal
Dec 14 13:25:08 rx-wj39106 systemd[1]: wbl.service: Main process exited, code=exited, status=1/FAILURE
Dec 14 13:25:08 rx-wj39106 systemd[1]: Stopped Binlog.
Dec 14 13:25:08 rx-wj39106 systemd[1]: wbl.service: Unit entered failed state.
Dec 14 13:25:08 rx-wj39106 systemd[1]: wbl.service: Failed with result 'exit-code'.

DROP TABLE支持还有点问题

看了正则，应该不能支持 DROP TABLE IF EXISTS t1;

->

expDropTable   = regexp.MustCompile("(?i)^DROP\\sTABLE(\\sIF\\sEXISTS){0,1}\\s.*?`{0,1}(.*?)`{0,1}\\.{0,1}`{0,1}([^`\\.]+?)`{0,1}($|\\s)")

binlog不支持的类型

不确定是否跟字段大小写有关。数据库不区分，但开发建表的时候，大小写都有。

$ tail -f log.txt
WARN[2017-12-04 12:50:46] binlog不支持的类型：CAR_OWNER_YEARS %!s()
WARN[2017-12-04 12:50:46] binlog不支持的类型：CAR_OWNER_WAY %!s()
WARN[2017-12-04 12:50:46] binlog不支持的类型：CAR_INSURANCE_INFO %!s()
WARN[2017-12-04 12:50:46] binlog不支持的类型：CAR_RELEASE_TIME %!s()
WARN[2017-12-04 12:50:46] binlog不支持的类型：HAS_RELEASE %!s()
WARN[2017-12-04 12:50:46] binlog不支持的类型：REGIST_CERT_TIME %!s()
WARN[2017-12-04 12:50:46] binlog不支持的类型：first_regist_date %!s()

DROP TABLE后cache没有处理

Step 1: 创建表：

CREATE TABLE dt_bool (dt_bit bit);

Step 2: 添加行：

INSERT INTO dt_bool VALUES (0);

Step 3: 收到信息：

2017-12-06 08:44:51 9次收到事件：{"database":"drafts","event":{"data":{"dt_bit":0},"event_type":"insert","time":1512521091},"event_index":16,"table":"dt_bool"}

Step 4: 删除重建表：

DROP TABLE dt_bool;
CREATE TABLE dt_bool (c BOOL);

Step 5: 添加行：

INSERT INTO dt_bool VALUES (1);

Step 6: 收到信息：
2017-12-06 08:44:53 10次收到事件：{"database":"drafts","event":{"data":{"dt_bit":1},"event_type":"insert","time":1512521093},"event_index":17,"table":"dt_bool"}

问题：注意收到的信息中的列名依然是dt_bit，应该是c。
解决：处理CREATE TABLE/DROP TABLE事件。

binlgo服务-SaveBinlogPostionCache缓冲区满

好像执行一个稍大的SQL文件，就会出现，不管有没有tcp连接。（允许了tcp/http）

WARN[2017-12-09 17:58:13] binlgo服务-SaveBinlogPostionCache缓冲区满...
WARN[2017-12-09 17:58:13] binlgo服务-SaveBinlogPostionCache缓冲区满...
WARN[2017-12-09 17:58:13] binlgo服务-SaveBinlogPostionCache缓冲区满...
WARN[2017-12-09 17:58:13] binlgo服务-SaveBinlogPostionCache缓冲区满...
WARN[2017-12-09 17:58:13] binlgo服务-SaveBinlogPostionCache缓冲区满...
WARN[2017-12-09 17:58:13] binlgo服务-SaveBinlogPostionCache缓冲区满...
WARN[2017-12-09 17:58:13] binlgo服务-SaveBinlogPostionCache缓冲区满...
WARN[2017-12-09 17:58:13] binlgo服务-SaveBinlogPostionCache缓冲区满...

Http接受这种SQL消息好像有问题

一次插入多条消息。

建表：

CREATE TABLE `foo` (
  `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `value` varchar(15) NOT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=23 DEFAULT CHARSET=utf8mb4;

执行的SQL：

INSERT INTO foo VALUES (NULL, 'Value-1'), (NULL, 'Value-2'), (NULL, 'Value-3');

Http服务器收到的数据：

[2017-12-05 15:03:56]	"{\"database\":\"drafts\",\"event\":{\"data\":{\"id\":22,\"value\":\"Value-3\"},\"event_type\":\"insert\",\"time\":1512457436},\"event_index\":6,\"table\":\"foo\"}"
[2017-12-05 15:03:56]	"{\"database\":\"drafts\",\"event\":{\"data\":{\"id\":22,\"value\":\"Value-3\"},\"event_type\":\"insert\",\"time\":1512457436},\"event_index\":6,\"table\":\"foo\"}"
[2017-12-05 15:03:56]	"{\"database\":\"drafts\",\"event\":{\"data\":{\"id\":22,\"value\":\"Value-3\"},\"event_type\":\"insert\",\"time\":1512457436},\"event_index\":6,\"table\":\"foo\"}"
[2017-12-05 15:03:56]	"{\"database\":\"drafts\",\"event\":{\"data\":{\"id\":22,\"value\":\"Value-3\"},\"event_type\":\"insert\",\"time\":1512457436},\"event_index\":6,\"table\":\"foo\"}"
[2017-12-05 15:03:56]	"{\"database\":\"drafts\",\"event\":{\"data\":{\"id\":22,\"value\":\"Value-3\"},\"event_type\":\"insert\",\"time\":1512457436},\"event_index\":6,\"table\":\"foo\"}"
[2017-12-05 15:03:56]	"{\"database\":\"drafts\",\"event\":{\"data\":{\"id\":22,\"value\":\"Value-3\"},\"event_type\":\"insert\",\"time\":1512457436},\"event_index\":6,\"table\":\"foo\"}"

log输出的时间好像不对

日志输出时间：

$ tail -f client.log 
2017-12-05 07:48:05 发送心跳包
2017-12-05 07:48:05 心跳包返回值：ok
2017-12-05 07:48:08 发送心跳包
2017-12-05 07:48:08 心跳包返回值：ok
2017-12-05 07:48:11 发送心跳包
2017-12-05 07:48:11 心跳包返回值：ok
2017-12-05 07:48:14 发送心跳包
2017-12-05 07:48:14 心跳包返回值：ok
2017-12-05 07:48:17 发送心跳包
2017-12-05 07:48:17 心跳包返回值：ok
2017-12-05 07:48:20 发送心跳包
2017-12-05 07:48:20 心跳包返回值：ok
2017-12-05 07:48:23 发送心跳包
2017-12-05 07:48:23 心跳包返回值：ok
2017-12-05 07:48:26 发送心跳包
2017-12-05 07:48:26 心跳包返回值：ok
2017-12-05 07:48:29 发送心跳包
2017-12-05 07:48:29 心跳包返回值：ok
^C

系统时间：

$ date
2017年 12月 05日 星期二 15:48:33 CST

好像是格林威治时间。

服务进程会卡住

src/library/binlog/binlog.go增加：

type handler struct {
	canal.DummyEventHandler
}

func (h *handler) OnRow(e *canal.RowsEvent) error {
	fmt.Println("================================================================================")
	fmt.Printf("Table:          %s\n", e.Table)
	fmt.Printf("Action:         %s\n", e.Action)
	fmt.Printf("Rows:           %v\n", e.Rows)
	for _, r := range e.Rows {
		k, err := canal.GetPKValues(e.Table, r)
		if err != nil {
			return nil
		}
		for _, v := range k {
			fmt.Printf("Primary key:    %+v\n", v)
		}
	}
	return nil
}

func (h *handler) OnDDL(p mysql.Position, e *replication.QueryEvent) error {
	fmt.Println("================================================================================")
	fmt.Printf("Binlog: %s : %d\n", p.Name, p.Pos)
	fmt.Printf("Database:       %s\n", string(e.Schema[:]))
	fmt.Printf("Query:          %s\n", string(e.Query[:]))
	fmt.Printf("Slave:          %d\n", e.SlaveProxyID)
	fmt.Printf("Execution Time: %d\n", e.ExecutionTime)
	fmt.Printf("Error:          %d\n", e.ErrorCode)
	return nil
}

func (h *handler) OnRotate(e *replication.RotateEvent) error {
	fmt.Println("================================================================================")
	log.Debugf("OnRotate event: ")
	return nil
}

func (h *handler) OnXID(p mysql.Position) error {
	fmt.Println("================================================================================")
	log.Debugf("OnXID event: ")
	return nil
}

func (h *handler) OnGTID(g mysql.GTIDSet) error {
	fmt.Println("================================================================================")
	log.Debugf("OnGTID event: ", g)
	return nil
}

func (h *handler) OnPosSynced(p mysql.Position, b bool) error {
	fmt.Println("================================================================================")
	log.Debugf("OnPosSynced event: %+v %b", p, b)
	return nil
}

func (h *handler) String() string {
	return "TestHandler"
}

并修改：
binlog.handler.SetEventHandler(&handler{})

则不会卡住，所以应该代码存在问题。

服务器：MariaDB 10.2.11 w/ CentOS 7.4

卡住是：服务没有任何错误和提示，tcp心跳仍然可以处理，在MariaDB主库SHOW PROCESSLIST; 看到复制进程状态一直 Wrting to net

这几个看是否可以修复，或者给go-mysql提交一个PR

正常关闭，不应该有错误提示。

ERRO[2017-12-16 15:36:19] canal start sync binlog err: context canceled
INFO[2017-12-16 15:36:19] wing-binlog-go service exit: context canceled
github.com/siddontang/go-mysql/canal/sync.go:55:
github.com/siddontang/go-mysql/canal/canal.go:203:
ERRO[2017-12-16 15:36:19] connection was bad
ERRO[2017-12-16 15:36:19] close sync with err: Sync was closed

[NFR] 多源复制

正常来说，线上会存在一台以上的主库，如果都接入程序的话，那就要多机或者多实例部署，而新版的MySQL/MariaDB都是可以支持多源复制的。（每个主库都有一个别名）

也就是说，如果程序可以支持多源复制类似的特性，在系统部署方面将即大的简化。

Websocket客户端断线，尝试重新连接

  1. 服务端启动
  2. 浏览器端启动
  3. 连接成功后，关闭服务器，模拟服务崩溃
  4. 服务器重新上线

观察一段时间浏览器console

2017-12-02 08:23:12 心跳返回值： ok
client.js:42:2
2017-12-02 08:23:14 客户端断线，尝试重新连接
client.js:42:2
InvalidStateError: An attempt was made to use an object that is not, or is no longer, usable
client.js:60
InvalidStateError: An attempt was made to use an object that is not, or is no longer, usable
client.js:60
InvalidStateError: An attempt was made to use an object that is not, or is no longer, usable
client.js:60
InvalidStateError: An attempt was made to use an object that is not, or is no longer, usable
client.js:60
InvalidStateError: An attempt was made to use an object that is not, or is no longer, usable
client.js:60
2017-12-02 08:23:14 设置注册分组返回值： ok
client.js:42:2
2017-12-02 08:23:15 心跳返回值： ok

一次会有多个InvalidStateError

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.