Code Monkey home page Code Monkey logo

bailongma's People

Contributors

anjia0532 avatar chenrulongmaster avatar fangpanpan avatar huskar-t avatar kim-up avatar liuyq-617 avatar sangshuduo avatar xiaopingcs avatar zitsen avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

bailongma's Issues

离线环境无法编译

/root/Bailongma-feature-openfalcon/blm_telegraf
[root@master2 blm_telegraf]# go build
go: github.com/emirpasic/[email protected]: Get "https://proxy.golang.org/github.com/emirpasic/gods/@v/v1.12.0.mod": dial tcp: lookup proxy.golang.org on 114.114.114.114:53: read udp 192.168.0.212:36019->114.114.114.114:53: i/o timeout
go: downloading github.com/taosdata/driver-go v0.0.0-20210722071233-99176d298e90
go: downloading github.com/emirpasic/gods v1.12.0
go: github.com/emirpasic/[email protected]: Get "https://proxy.golang.org/github.com/emirpasic/gods/@v/v1.12.0.mod": dial tcp: lookup proxy.golang.org on 114.114.114.114:53: read udp 192.168.0.212:36019->114.114.114.114:53: i/o timeout
[root@master2 blm_telegraf]# ls
Dockerfile  go.mod  go.sum  README.md  server.go  Telegraf写入TDengine的API.md
[root@master2 blm_telegraf]# go version
go version go1.17.1 linux/amd64

工程编译问题

HI:
如果可以的话,帮忙在一个干净环境里重新编译测试看看。
1、go.mod是不是可以直接写在工程里?而不是用go mod init
2、默认情况下promethues的版本不存在prompb
3、需要依赖TDengine,但是TDengine, submodule clone问题,这里存在git clone权限不足
4、解决了prompb module问题,但是传入promb变量有问题,这里估计是prompb版本问题

谢谢。

Bailongma can not connect to taos db cluster in kubernetes

Hi, @sangshuduo
I deployed an tdengine cluster in k8s followed TDengine-Operator, following is my tdengine pods and service, and it worked.
image

then i deployed bailongma
image

bailongma can not connect to taos. following is logs
image

and i can connect to tdengine by restful api in blm_prometheus container.

image

编译 blm_prometheus 报错

root@dcim:/opt/Bailongma# go mod tidy
go: finding module for package github.com/emirpasic/gods/sets/hashset
go: finding module for package github.com/prometheus/common/model
go: finding module for package github.com/prometheus/prometheus/prompb
go: finding module for package github.com/taosdata/driver-go/taosSql
go: finding module for package github.com/golang/snappy
go: finding module for package github.com/gogo/protobuf/proto
go: found github.com/gogo/protobuf/proto in github.com/gogo/protobuf v1.3.2
go: found github.com/golang/snappy in github.com/golang/snappy v0.0.4
go: found github.com/prometheus/common/model in github.com/prometheus/common v0.31.1
go: found github.com/prometheus/prometheus/prompb in github.com/prometheus/prometheus v2.5.0+incompatible
go: found github.com/emirpasic/gods/sets/hashset in github.com/emirpasic/gods v1.12.0
go: finding module for package github.com/grpc-ecosystem/grpc-gateway/utilities
go: finding module for package github.com/taosdata/driver-go/taosSql
go: finding module for package github.com/grpc-ecosystem/grpc-gateway/runtime
go: found github.com/grpc-ecosystem/grpc-gateway/runtime in github.com/grpc-ecosystem/grpc-gateway v1.16.0
go: found github.com/grpc-ecosystem/grpc-gateway/utilities in github.com/grpc-ecosystem/grpc-gateway v1.16.0
go: finding module for package github.com/taosdata/driver-go/taosSql
bailongma/v2/blm_prometheus imports
        github.com/taosdata/driver-go/taosSql: no matching versions for query "latest"
root@dcim:/opt/Bailongma/blm_prometheus# go get github.com/taosdata/driver-go/taosSql
go get github.com/taosdata/driver-go/taosSql: no matching versions for query "upgrade"

项目 github.com/taosdata/driver-go 已发布 v2 版本,详见:taosdata/driver-go

启动报错 connection refused

./blm_prometheus -port 8088 启动,终端直接报错
Get "http://172.23.11.4:6020/rest/sql": dial tcp 172.23.11.4:6020: connect: connection refused Get "http://172.23.11.4:6020/rest/sql": dial tcp 172.23.11.4:6020: connect: connection refused Get "http://172.23.11.4:6020/rest/sql": dial tcp 172.23.11.4:6020: connect: connection refused Get "http://172.23.11.4:6020/rest/sql": dial tcp 172.23.11.4:6020: connect: connection refused Get "http://172.23.11.4:6020/rest/sql": dial tcp 172.23.11.4:6020: connect: connection refused Get "http://172.23.11.4:6020/rest/sql": dial tcp 172.23.11.4:6020: connect: connection refused Get "http://172.23.11.4:6020/rest/sql": dial tcp 172.23.11.4:6020: connect: connection refused Get "http://172.23.11.4:6020/rest/sql": dial tcp 172.23.11.4:6020: connect: connection refused

查看taosd log 如下:
06/08 20:35:32.069964 00023174 MND ERROR msg:0x7f6e44000d60, app:0x66a table:0.prometheus.mysql_info_schema_table_size, failed to get table meta, table not exist 06/08 20:35:32.070581 00023175 MND ERROR msg:0x7f6e44000d60, app:0x66c table:0.prometheus.mysql_info_schema_table_size, failed to get table meta, table not exist 06/08 20:35:32.071263 00023174 MND ERROR msg:0x7f6e44000d60, app:0x66e table:0.prometheus.mysql_info_schema_table_size, failed to get table meta, table not exist 06/08 20:35:32.071779 00023175 MND ERROR msg:0x7f6e44000d60, app:0x670 table:0.prometheus.mysql_info_schema_table_size, failed to get table meta, table not exist 06/08 20:35:32.072324 00023174 MND ERROR msg:0x7f6e44000d60, app:0x672 table:0.prometheus.mysql_info_schema_table_size, failed to get table meta, table not exist 06/08 20:35:32.072893 00023175 MND ERROR msg:0x7f6e44000d60, app:0x674 table:0.prometheus.mysql_info_schema_table_size, failed to get table meta, table not exist 06/08 20:35:32.073430 00023174 MND ERROR msg:0x7f6e44000d60, app:0x676 table:0.prometheus.mysql_info_schema_table_size, failed to get table meta, table not exist 06/08 20:35:33.020489 00023174 MND ERROR msg:0x7f6e44000d60, app:0x678 table:0.prometheus.md5_ed032e52a5b5bb7a4b190fad66074a46, failed to get table meta, table not exist

文档说库和表会自动创建,但实际log显示都没创建

The file "server.go" can't build in TDengine 2.0.3.1 (blm_promepheus)

I want to rebuild the project as directed by the link:https://www.taosdata.com/cn/documentation/insert/#Prometheus直接写入
But I cannot go build, and get ERR:

execSql Error: syntax error near ")
" sqlcmd: create table if not exists MD5_648e4536f5d4184314287d187677867c using node_network_address_assign_type tags("623716d73194f57a558f0d1cb4964378",)

I modify the file "server.go" and add a line after line 529:

line 529: sqlcmd = sqlcmdhead + md5V2(tagHash) + ""," + sqlcmd
line 530: sqlcmd = strings.Replace(sqlcmd, ",)\n", ")\n", -1)

But it still cannot run normally after modification. Attached is the generated error log, please help update the code or indicate whether there is a problem in my configuration.
This is the table of project configuration:

ServerName IP Port APP
taos1 10.197.190.17 6030~6042 Tdengine 2.0.3.1
taos2 10.197.190.18 6030~6042 Tdengine 2.0.3.1
taos3 10.197.190.19 6030~6042 Tdengine 2.0.3.1
app 10.197.190.20 3000 Grafana 7.1.5
Telegraf
6051 blm_telegraf
Prometheus 10.197.190.21 9090 Prometheus
6052 blm_prometheus

blm_prometheus.log
prometheus.yml.log
server.go.log
taosdlog.0.log

prometheus 配置了BLM 没有数据写到TDengine,也没有报错

问题:prometheus 配置了BLM 没有数据写到TDengine,也没有报错
prometheus 版本:2.24.0
TDengine 版本:2.0.15

1、prometheus的启动日志:
image
2、prometheus的配置:
image
3、BIM服务也是正常的,在安装了prometheus的机器上,直接http请求:
image
4、BLM的启动参数,并且进程也在的:
image
5、然后TDengine的机器上查不到数据,连数据库都没有创建:
image
6、实际在grafana里面是有数据写入prometheus的:
image

blm_prometheus 可能存在内存泄漏,使用内存一直在增长

使用 bailongma master 以及 prometheus v2.23.0 编译,修改部分语法错误(prometheus v2.23.0同v2.5.0部分struct不兼容) 运行后,内存持续增长( v2.5.0 也有同样的问题,所以才升级到 v2.23.0 测试)

运行: go tool pprof 'http://localhost:8080/debug/pprof/heap'

File: blm_prometheus
Build ID: 1d27bd0ba96f37057ab7eb9aa9f5e516fd4b7cf2
Type: inuse_space
Time: Dec 24, 2020 at 5:31pm (CST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 1370.07MB, 97.38% of 1406.88MB total
Dropped 32 nodes (cum <= 7.03MB)
Showing top 10 nodes out of 31
flat flat% sum% cum cum%
690.07MB 49.05% 49.05% 1001.09MB 71.16% github.com/prometheus/prometheus/prompb.(*TimeSeries).Unmarshal
311.02MB 22.11% 71.16% 311.02MB 22.11% github.com/prometheus/prometheus/prompb.(*Label).Unmarshal
166.04MB 11.80% 82.96% 166.04MB 11.80% bufio.NewWriterSize (inline)
114.43MB 8.13% 91.09% 114.43MB 8.13% bufio.NewReaderSize
52.51MB 3.73% 94.82% 1053.59MB 74.89% github.com/prometheus/prometheus/prompb.(*WriteRequest).Unmarshal
9.50MB 0.68% 95.50% 9.50MB 0.68% runtime.malg
9.50MB 0.68% 96.18% 12.50MB 0.89% net/textproto.(*Reader).ReadMIMEHeader
7.50MB 0.53% 96.71% 92.10MB 6.55% net/http.(*conn).readRequest
7MB 0.5% 97.21% 27.50MB 1.95% net/http.readRequest
2.50MB 0.18% 97.38% 1056.09MB 75.07% main.main.func1

过一段时间后,运行

File: blm_prometheus
Build ID: 1d27bd0ba96f37057ab7eb9aa9f5e516fd4b7cf2
Type: inuse_space
Time: Dec 24, 2020 at 5:38pm (CST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 1743.78MB, 97.58% of 1787.07MB total
Dropped 34 nodes (cum <= 8.94MB)
Showing top 10 nodes out of 31
flat flat% sum% cum cum%
872.86MB 48.84% 48.84% 1262.88MB 70.67% github.com/prometheus/prometheus/prompb.(*TimeSeries).Unmarshal
390.02MB 21.82% 70.67% 390.02MB 21.82% github.com/prometheus/prometheus/prompb.(*Label).Unmarshal
212.18MB 11.87% 82.54% 212.18MB 11.87% bufio.NewWriterSize (inline)
147.56MB 8.26% 90.80% 147.56MB 8.26% bufio.NewReaderSize
66.65MB 3.73% 94.53% 1329.53MB 74.40% github.com/prometheus/prometheus/prompb.(*WriteRequest).Unmarshal
15MB 0.84% 95.37% 18MB 1.01% net/textproto.(*Reader).ReadMIMEHeader
14.51MB 0.81% 96.18% 14.51MB 0.81% runtime.malg
12MB 0.67% 96.85% 126.14MB 7.06% net/http.(*conn).readRequest
9MB 0.5% 97.35% 37.51MB 2.10% net/http.readRequest
4MB 0.22% 97.58% 1333.53MB 74.62% main.main.func1

系统运行环境:

Linux xxx 3.10.0-1127.19.1.el7.x86_64 #1 SMP Tue Aug 25 17:23:54 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
CentOS Linux release 7.9.2009 (Core)
go version go1.15.2 linux/amd64

启动后运行一会儿报错

panic: Checktable error value type

goroutine 22 [running]:
main.SerilizeTDengine(0xc00059c5d0, 0xc0005956d0, 0xc, 0xc00059c600, 0x177289bb8b0, 0x736fdd, 0x8, 0xc000210000, 0xe, 0xc0005eb050, ...)
/root/blm_telegraf/server.go:346 +0xc6a
main.ProcessData(0xc0000f27e0, 0x1, 0x1, 0x736fdd, 0x8, 0xc000210000, 0xe, 0x0, 0x0)
/root/blm_telegraf/server.go:392 +0x1030
main.ProcessReq(0xc000580000, 0x4d2, 0x63f, 0xc000210000, 0xe, 0x0, 0x0)
/root/blm_telegraf/server.go:284 +0x3d5
main.NodeProcess(0x4, 0x8, 0x9)
/root/blm_telegraf/server.go:210 +0xce
created by main.main
/root/blm_telegraf/server.go:139 +0x164
host: tdengine:0 port: 10202 database: telegraf%

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.