i downloaded mysqlbeat from https://github.com/adibendahan/mysqlbeat/releases/download/1.0.0/mysqlbeat_1.0.0-160512235547_amd64.deb and installed it on my system(Ubuntu 19.04). after this ,I started it.
But according to my observation，the mysqlbeat service will restart per 10s. why?is there something iconfiged error?

./main.go:12: cannot use beater.New() (type *beater.Mysqlbeat) as type beat.Creator in argument to beat.Run

github.com/adibendahan/mysqlbeat/beater

beater/mysqlbeat.go:289: b.Events undefined (type *beat.Beat has no field or method Events)
beater/mysqlbeat.go:303: b.Events undefined (type *beat.Beat has no field or method Events)
beater/mysqlbeat.go:326: b.Events undefined (type *beat.Beat has no field or method Events)

Hi i have a sql like select t1 a , b from t1, t2 where...

mysql db has 2 rows.
But After mysqlbeat indexded into ES. it has 8 document .

which every mysql row has 4 times in ES. only different id!

How can do for fix this.
Thanks.

From what I see, I encountered the same thing on #10. and got the complete output as follows:

root@elknode1:/home/vagrant/mysqlbeat# make
go build
# _/home/vagrant/mysqlbeat
./main.go:12: cannot use beater.New() (type *beater.Mysqlbeat) as type "github.com/elastic/beats/libbeat/beat".Beater in argument to "github.com/elastic/beats/libbeat/beat".Run:
        *beater.Mysqlbeat does not implement "github.com/elastic/beats/libbeat/beat".Beater (wrong type for Cleanup method)
                have Cleanup(*"github.com/adibendahan/mysqlbeat/vendor/github.com/elastic/beats/libbeat/beat".Beat) error
                want Cleanup(*"github.com/elastic/beats/libbeat/beat".Beat) error
vendor/github.com/elastic/beats/libbeat/scripts/Makefile:56: recipe for target 'build' failed
make: *** [build] Error 2

The current version I have is the recent one:

root@elknode1:/home/vagrant/mysqlbeat# filebeat version
filebeat version 6.5.4 (amd64), libbeat 6.5.4 [bd8922f1c7e93d12b07e0b3f7d349e17107f7826 built 2018-12-17 20:22:29 +0000 UTC]

and go version is

root@elknode1:/home/vagrant/mysqlbeat# go version
go version go1.7.4 linux/amd64

Is this still being maintained since the last update was almost 2 years ago.

ask for more installation informatiion.

I am not able to install mysqlbeat. I have made sure that gopath is set.

[root@localhost mysqlbeat]# make
go build
main.go:8:2: cannot find package "github.com/adibendahan/mysqlbeat/beater" in any of:
/home/go/src/github.com/adibendahan/mysqlbeat/beater (from $GOROOT)
($GOPATH not set. For more details see: 'go help gopath')
main.go:6:2: cannot find package "github.com/elastic/beats/libbeat/beat" in any of:
/home/go/src/github.com/elastic/beats/libbeat/beat (from $GOROOT)
($GOPATH not set. For more details see: 'go help gopath')
make: *** [build] Error 1

After completed the mysqlbeat installation when I try to view the new dashboards or its related visualizations for the mysqlbeat I get the following error:
Saved "field" parameter is now invalid. Please select a new field.
Visualize: "field" is a required parameter

finally I decided to focus on the visualization "MySQL-InnoDB-I-slash-O" in order to try to figure out what is the problem:
so at the file mysql_performance_dashboard_by_mysqlbeat.json I see the following definition:
{
"_id": "MySQL-InnoDB-I-slash-O",
"_type": "visualization",
"_source": {
"title": "MySQL - InnoDB I/O",
"visState": "{"title":"MySQL - InnoDB Buffer Pool Usage","type":"area","params":{"shareYAxis":true,"addTooltip":true,"addLegend":true,"smoothLines":true,"scale":"linear","interpolate":"linear","mode":"stacked","times":[],"addTimeMarker":false,"defaultYExtents":false,"setYExtents":false,"yAxis":{}},"aggs":[{"id":"2","type":"terms","schema":"split","params":{"field":"beat.name","size":10,"order":"asc","orderBy":"_term","row":true}},{"id":"3","type":"date_histogram","schema":"segment","params":{"field":"@timestamp","interval":"auto","customInterval":"2h","min_doc_count":1,"extended_bounds":{},"customLabel":"Datetime"}},{"id":"5","type":"max","schema":"metric","params":{"field":"INNODB_DATA_READS_PERSECOND","customLabel":"Data Reads"}},{"id":"4","type":"max","schema":"metric","params":{"field":"INNODB_DATA_WRITES_PERSECOND","customLabel":"Data Writes"}},{"id":"1","type":"max","schema":"metric","params":{"field":"INNODB_LOG_WRITES_PERSECOND","customLabel":"Log Writes"}},{"id":"6","type":"max","schema":"metric","params":{"field":"INNODB_DATA_FSYNCS_PERSECOND","customLabel":"Data fsyncs"}},{"id":"7","type":"max","schema":"metric","params":{"field":"INNODB_OS_LOG_FSYNCS_PERSECOND","customLabel":"OS Log fsyncs"}}],"listeners":{}}",
"uiStateJSON": "{}",
"description": "",
"savedSearchId": "mysqlbeat",
"version": 1,
"kibanaSavedObjectMeta": {
"searchSourceJSON": "{"filter":[]}"
}
}
},
but at the mysqlbeat.yml I don't see where a queries contain the fields INNODB_LOG_WRITES_PERSECOND, INNODB_DATA_FSYNCS_PERSECON, INNODB_OS_LOG_FSYNCS_PERSECOND
Because it convert those fields name to a name with a surffix '__DELTA' and no where select that columns with the suffix '_PERSECON'

queries: [ "SELECT CONCAT(VARIABLE_NAME, '__DELTA') AS VARIABLE_NAME, VARIABLE_VALUE
FROM information_schema.GLOBAL_STATUS
WHERE VARIABLE_NAME IN ('ABORTED_CONNECTS', 'CONNECTION_ERRORS_INTERNAL', 'CONNECTION_ERRORS_MAX_CONNECTIONS', 'CREATED_TMP_DISK_TABLES', 'CREATED_TMP_FILES',
'CREATED_TMP_TABLES', 'HANDLER_COMMIT', 'HANDLER_ROLLBACK', 'HANDLER_SAVEPOINT', 'HANDLER_SAVEPOINT_ROLLBACK', 'INNODB_DATA_FSYNCS',
'INNODB_DATA_READS', 'INNODB_DATA_WRITES', 'INNODB_LOG_WRITES', 'INNODB_OS_LOG_FSYNCS', 'INNODB_PAGES_CREATED', 'INNODB_PAGES_READ',
'INNODB_PAGES_WRITTEN', 'INNODB_ROWS_DELETED', 'INNODB_ROWS_INSERTED', 'INNODB_ROWS_READ', 'INNODB_ROWS_UPDATED', 'INNODB_ROW_LOCK_TIME',
'KEY_READS', 'KEY_READ_REQUESTS', 'KEY_WRITES', 'KEY_WRITE_REQUESTS', 'OPENED_TABLES', 'QCACHE_HITS', 'QCACHE_INSERTS', 'QCACHE_LOWMEM_PRUNES',
'QCACHE_NOT_CACHED', 'SELECT_FULL_JOIN', 'SELECT_FULL_RANGE_JOIN', 'SELECT_RANGE', 'SELECT_RANGE_CHECK', 'SELECT_SCAN', 'SLOW_QUERIES',
'SORT_MERGE_PASSES', 'SORT_RANGE', 'SORT_ROWS', 'SORT_SCAN', 'TABLE_LOCKS_IMMEDIATE', 'TABLE_LOCKS_WAITED', 'INNODB_ROW_LOCK_WAITS')
UNION
SELECT VARIABLE_NAME, VARIABLE_VALUE
FROM information_schema.GLOBAL_VARIABLES
WHERE VARIABLE_NAME IN ('MAX_CONNECTIONS', 'TABLE_OPEN_CACHE')
UNION
SELECT VARIABLE_NAME, VARIABLE_VALUE
FROM information_schema.GLOBAL_STATUS
WHERE VARIABLE_NAME IN ('INNODB_BUFFER_POOL_PAGES_DATA', 'INNODB_BUFFER_POOL_PAGES_FREE', 'INNODB_BUFFER_POOL_PAGES_TOTAL', 'OPEN_FILES', 'OPEN_TABLES',
'QCACHE_QUERIES_IN_CACHE', 'THREADS_CONNECTED', 'THREADS_RUNNING', 'INNODB_PAGE_SIZE')
UNION
SELECT 'INNODB_BUFFER_POOL_HIT_RATIO' AS VARIABLE_NAME,
INNODB_BUFFER_POOL_READ_REQUESTS.VARIABLE_VALUE / (INNODB_BUFFER_POOL_READ_REQUESTS.VARIABLE_VALUE + INNODB_BUFFER_POOL_READS.VARIABLE_VALUE) * 100 AS VARIABLE_VALUE
FROM information_schema.GLOBAL_STATUS INNODB_BUFFER_POOL_READS, information_schema.GLOBAL_STATUS INNODB_BUFFER_POOL_READ_REQUESTS
WHERE INNODB_BUFFER_POOL_READS.variable_name ='INNODB_BUFFER_POOL_READS' AND
INNODB_BUFFER_POOL_READ_REQUESTS.variable_name ='INNODB_BUFFER_POOL_READ_REQUESTS'
UNION
SELECT 'CONST_100' AS VARIABLE_NAME, 100 AS VARIABLE_VALUE",
"SHOW SLAVE STATUS"]

From where does it suppose to get the columns with the suffix '_PERSECON'?

Thank you,
Ofir

Total Neophyte with Go. Need to change the periodicity of mysql queries such that the contents of the mysql table are pulled once every 24 hours (e.g. SELECT * from table WHERE etl_date is...).

It seems that the delay from period parameter is counted down prior to the first iteration? So when setting period to 86,400 it is waiting for 24 hours before executing the first iteration. Is this correct? How would you change the code to execute the first query prior to starting a delay loop?

Thanks in advance!

Hello,

I am currently trying to use your custom beat. When trying to pull data from my DB everything runs smoothly, except for one thing. I can't figure out how to only update my indexes only with new or modified content. I considered these queries :

queries: [ "SELECT THE_ID,
   THE_USER, 
   THE_DATE FROM    thetable.USER"]

This one doesnt work for obvious reasons, but these two don't either :

queries: [ "SELECT THE_ID, 
    THE_USER, 
    CONCAT(THE_DATE, '__DELTA') AS THE_DATE 
    FROM    thetable.USER"]

queries: [ "SELECT CONCAT(THE_ID, '__DELTAKEY), 
    THE_USER, 
    CONCAT(THE_DATE, '__DELTAKEY') AS THE_DATE 
    FROM    thetable.USER"]

I was hoping maybe you could help me out on this issue, wether that's because I misundestrood your documentation, or you didn't incorporate these kind of delta's in the beat.

Thanks !

When running mysqlbeat, the command fails to push the data to elasticsearch. In the logs, I could see below errror
ERR Failed to perform any bulk index operations: 406 Not Acceptable
2018-06-26T13:50:39+05:30 INFO Error publishing events (retrying): 406 Not Acceptable

Some are not accurate or not getting the right data..

Sample:
QCACHE_HITS is getting only 1, but it should be 700000+..
Is this a problem with the _DELTA change from PERSECOND?

I am trying to run mysqlbeat in centos and the command "mysqlbeat -c custom.yml" hangs forever without displaying any log status.. Is there a place I can look for the full stack trace and find why this is happening ?

Hello adibendahan,

    I change mysqlbeat.yml and config hostname username password and so on 

But I don't know how to change dbname for my sql query. could you help me? thanks.

stan@ubuntuos:~/gowork/src/github.com/adibendahan/mysqlbeat$ ./mysqlbeat -c mysqlbeat.yml
Start error: Error 1046: No database selected

Please correct me if I am wrong, it seems that the visualization that the mysql_performance_dashboard_by_mysqlbeat.json dashboard find is not located on the source code.

I want to use this beat to pull the same queries from databases in multiple regions.

Receive the following error while you try to connect to a server that forces TLS.
Start error: Error 3159: Connections using insecure transport are prohibited while --require_secure_transport=ON.

when i run make command on under mysqlbeat folder error happens
go build

github.com/adibendahan/mysqlbeat/beater

../github.com/adibendahan/mysqlbeat/beater/mysqlbeat.go:474: b.Events undefined (type beat.Beat has no field or method Events)
../github.com/adibendahan/mysqlbeat/beater/mysqlbeat.go:503: b.Events undefined (type *beat.Beat has no field or method Events)
make: ** [build] Error 2

Hello,I'm stuck with a problem when I use it
I am using
Elastic search - 6.2.3
Logstash - 6.2.3
Kibana - 6.2.3.
I use it mysqlbeat-1.0.0-160512235547-x86_64.rpm installation. After configuration startup：
May 16 11:42:51 dev05 /usr/bin/mysqlbeat[44583]: client.go:136: Failed to perform any bulk index operations: 406 Not Acceptable
May 16 11:43:51 dev05 /usr/bin/mysqlbeat[44583]: client.go:136: Failed to perform any bulk index operations: 406 Not Acceptable
May 16 11:44:51 dev05 /usr/bin/mysqlbeat[44583]: client.go:136: Failed to perform any bulk index operations: 406 Not Acceptable
May 16 11:45:52 dev05 /usr/bin/mysqlbeat[44583]: client.go:136: Failed to perform any bulk index operations: 406 Not Acceptable

I don't know what the cause is.
I configure the log path, but not. This is seen in the system log
Request you to advice! I am really stuck!
Thanks in advance!

When I added a new query, I got this error.

Start error: setup returned an error: error on config file, queries array length != queryTypes array length (each query should have a corresponding type on the same index)

How do I configure the template to include the results of the new query?