Code Monkey home page Code Monkey logo

clickvisual's Introduction

ClickVisual

GitHub stars GitHub issues GitHub license Release Go Report Card go.dev reference All Contributors

English | 中文

ClickVisual is a lightweight browser-based logs analytics and logs search platform for ClickHouse.

Documentation

See https://clickvisual.net

Log Query Demonstration

log-search

Alarm Process Demonstration

log-search

DAG Workflow

log-search

Configuration Page

log-search

Features

  • Support visual query dashboard, query histogram and raw logs for SQL.
  • Support showing percentage for specified fields.
  • Support vscode style configuration board, you can easily emit your fluent-bit configuration to Kubernetes ConfigMap.
  • Out of the box, easily deployment with kubectl.
  • Support for GitHub and GitLab Authentication.

Architecture

image

Installation

  • For Docker
# clone clickvisual source code.
git clone https://github.com/clickvisual/clickvisual.git

# you may need to set docker image mirror, visit <https://github.com/yeasy/docker_practice/blob/master/install/mirror.md> for details.
docker-compose up

# then go to browser and visit http://localhost:19001.
# login username: clickvisual 
# login password: clickvisual
  • For host
# download release.
# get latest version.
latest=$(curl -sL https://api.github.com/repos/clickvisual/clickvisual/releases/latest | grep  ".tag_name" | sed -E 's/.*"([^"]+)".*/\1/')

# for MacOS amd64.
wget "https://github.com/clickvisual/clickvisual/releases/download/${latest}/clickvisual-${latest}-darwin-amd64.tar.gz" -O clickvisual-${latest}.tar.gz 

# for Linux amd64.
wget "https://github.com/clickvisual/clickvisual/releases/download/${latest}/clickvisual-${latest}-linux-amd64.tar.gz" -O clickvisual-$(latest).tar.gz  

# extract zip file to current directory.
mkdir -p ./clickvisual-${latest} && tar -zxvf clickvisual-${latest}.tar.gz -C ./clickvisual-${latest}

# open config/default.toml, then change database and redis or other section configuration
# execute migration latest sql script in scripts/migration directory
# start clickvisual
cd ./clickvisual-${latest} && ./clickvisual -config config/default.toml

# then go to browser and visit http://localhost:19001
# login username: clickvisual
# login password: clickvisual

Document Contribution

If you want to participate in https://clickvisual.net document updating activities
Please refer to this document https://github.com/clickvisual/clickvisual/tree/master/docs

Join Us

Join us, please add the "cv" keyword in the verification information.

Wechat id is "MEXES_"

Contributors

Thanks for these wonderful people:


MEX7

m1666

askuy

sevennt

LincolnZhou

Link Duan

梁桂锋

qingbozhang

qianque7

Chen Ziqian

antony

ArthurQ

Jeff Li

Ather Shu

Jeremy

csy

zackzhangkai

kl

Thank You

Thank JetBrains for Open Source licenses support

JetBrains

Friends

clickvisual's People

Contributors

allcontributors[bot] avatar antonyaz avatar arthurqiuys avatar askuy avatar dependabot[bot] avatar fangpengbo avatar hantmac avatar isyanthony avatar kl7sn avatar klboke avatar laojianzi avatar liangguifeng avatar link-duan avatar m1666 avatar pigcsy avatar qianque7 avatar qingbozhang avatar rotk2022 avatar sevennt avatar shushenghong avatar testwill avatar xiedeyantu avatar zackzhangkai avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

clickvisual's Issues

help wanted: 帮忙登记下你的信息 / Help to register your information

欢迎使用 ClickVisual,首先感谢你的使用,其次您可以参考下面的样例来提供您的信息以收集下使用场景,该信息只会用于ClickVisual 社区内的宣传,再次感谢你对该项目的支持:

组织:ClickVisual(Required)
图片:公司 logo(Required)
地点:**武汉(Required)
联系方式:邮箱或官方网站(Optional)
场景:作为业务日志、大数据行为分析使用(Required)
再次感谢你的参与!!!

Welcome to use ClickVisual To know who is using it, pls append your org info as follow.
This infomation will only be used for promotion within the ClickVisual community.
Thank you again for your support of the project.

Organization:ClickVisual (Required)
Logo: Company Logo (Required)
Location: Wuhan, China(Required)
Contact: Email or Official website (Optional)
Purpose:use as our biz logger ui (Required)

Thanks again for your participation!

how to keep mogo simple & stable while running by docker-compose up all-in-one method?

when install by "For Docker" method as official doc , the command line termial came out with so many info& error that I can't determine which is the final cause of top-ui-presented-problem, In my case, the web request to http://localhost:19001 will unstoppablly 301-redirect to http://localhost:19001/user/login/?referer=http%3A%2F%2Flocalhost%3A19001%2Fuser%2Flogin%2F%3Freferer%3Dhttp%253A%252F%252Flocalhost%253A19001%252Fuser%252Flogin%252F%253Freferer%253Dhttp%25253A%25252F%25252Flocalhost%25253A19001%25252Fuser%25252Flogin%25252F%25253Freferer%25253Dhttp%2525253A%2525252F%2525252Flocalhost%2525253A19001%2525252Fuser%2525252Flogin%2525252F%2525253Freferer%2525253Dhttp%252525253A%252525252F%252525252Flocalhost%252525253A19001%252525252Fuser%252525252Flogin%252525252F%252525253Freferer%252525253Dhttp%25252525253A%25252525252F%25252525252Flocalhost%25252525253A19001%25252525252Fuser%25252525252Flogin%25252525252F%25252525253Freferer%25252525253Dhttp%2525252525253A%2525252525252F%2525252525252Flocalhost%2525252525253A19001%2525252525252Fuser%2525252525252Flogin%2525252525252F%2525252525253Freferer%2525252525253Dhttp%252525252525253A%252525252525252F%252525252525252Flocalhost%252525252525253A19001%252525252525252Fuser%252525252525252Flogin%252525252525252F%252525252525253Freferer%252525252525253Dhttp%25252525252525253A%25252525252525252F%25252525252525252Flocalhost%25252525252525253A19001%25252525252525252Fuser%25252525252525252Flogin%25252525252525252F%25252525252525253Freferer%25252525252525253Dhttp%2525252525252525253A%2525252525252525252F%2525252525252525252Flocalhost%2525252525252525253A19001%2525252525252525252Fuser%2525252525252525252Flogin%2525252525252525252F%2525252525252525253Freferer%2525252525252525253Dhttp%252525252525252525253A%252525252525252525252F%252525252525252525252Flocalhost%252525252525252525253A19001%252525252525252525252Fuser%252525252525252525252Flogin%252525252525252525252F%252525252525252525253Freferer%252525252525252525253Dhttp%25252525252525252525253A%25252525252525252525252F%25252525252525252525252Flocalhost%25252525252525252525253A19001%25252525252525252525252Fuser%25252525252525252525252Flogin%25252525252525252525252F%25252525252525252525253Freferer%25252525252525252525253Dhttp%2525252525252525252525253A%2525252525252525252525252F%2525252525252525252525252Flocalhost%2525252525252525252525253A19001%2525252525252525252525252Fuser%2525252525252525252525252Flogin%2525252525252525252525252F%2525252525252525252525253Freferer%2525252525252525252525253Dhttp%252525252525252525252525253A%252525252525252525252525252F%252525252525252525252525252Flocalhost%252525252525252525252525253A19001%252525252525252525252525252F.

If docker-compose.yml can make the out put message not so massive, and only output the error or the only-necessary message. Or it's hard to find out the true cause .

Here is part of the output:

mock_1          | 
mock_1          |     _/_/_/_/    _/_/_/    _/_/   
mock_1          |    _/        _/        _/    _/   
mock_1          |   _/_/_/    _/  _/_/  _/    _/    
mock_1          |  _/        _/    _/  _/    _/     
mock_1          | _/_/_/_/    _/_/_/    _/_/  
mock_1          | 
mock_1          | Welcome to Ego, starting application ...
mock_1          | 
mock_1          | {"lv":"info","ts":1646783004,"msg":"init config","lname":"ego.sys","comp":"core.econf","addr":"/mogo/config/docker.toml"}
mock_1          | {"lv":"info","ts":1646783004,"msg":"read watch","lname":"ego.sys","comp":"core.econf","comp":"file datasource","configFile":"/mogo/config/docker.toml","realConfigFile":"/mogo/config/docker.toml","fppath":"/mogo/config/docker.toml"}
mock_1          | {"lv":"info","ts":1646783004,"msg":"init app","lname":"ego.sys","comp":"app","pid":1,"coreNum":2}
mock_1          | {"lv":"info","ts":1646783004,"msg":"reinit default logger","lname":"ego.sys","comp":"core.elog"}
mock_1          | {"lv":"info","ts":1646783004,"msg":"init trace","lname":"ego.sys","comp":"app"}
mock_1          | {"lv":"info","ts":1646783004,"msg":"start db","lname":"ego.sys","comp":"component.egorm","compName":"mysql.default","addr":"mysql:13306","name":"mocro"}
mock_1          | {"lv":"debug","ts":1646783004,"msg":"ClickHouse","lname":"default.log","step":"ch process","dsn":""}
mock_1          | [clickhouse]host(s)=clickhouse:9000, database=default, username=root
mock_1          | [clickhouse][dial] secure=false, skip_verify=false, strategy=random, ident=1, server=0 -> 172.21.0.9:9000
mock_1          | [clickhouse][connect=1][hello] -> Golang SQLDriver 1.1.54213
mock_1          | {"lv":"debug","ts":1646783004,"msg":"ClickHouse","lname":"default.log","step":"ch finish"}
mock_1          | [clickhouse][connect=1][hello] <- ClickHouse 22.2.54455 (UTC)
mock_1          | [clickhouse][connect=1]-> ping
mock_1          | [clickhouse][connect=1][process] <- pong
mock_1          | {"lv":"info","ts":1646783004,"msg":"AuthInit","lname":"default.log","step":"UnmarshalKey","oauthInfos":[{"Typ":"github","ClientId":"","ClientSecret":"","Scopes":["user:email","read:org"],"AuthUrl":"https://github.com/login/oauth/authorize","TokenUrl":"https://github.com/login/oauth/access_token","Enable":true,"EmailAttributeName":"","EmailAttributePath":"","RoleAttributePath":"","AllowedDomains":[],"HostedDomain":"","ApiUrl":"https://api.github.com/user","AllowSignup":true,"Name":"","TlsClientCert":"","TlsClientKey":"","TlsClientCa":"","TlsSkipVerify":false,"TeamIds":[],"AllowedOrganizations":[],"AllowedGroups":null},{"Typ":"gitlab","ClientId":"","ClientSecret":"","Scopes":["api"],"AuthUrl":"https://gitlab.com/oauth/authorize","TokenUrl":"https://gitlab.com/oauth/token","Enable":true,"EmailAttributeName":"","EmailAttributePath":"","RoleAttributePath":"","AllowedDomains":[],"HostedDomain":"","ApiUrl":"https://gitlab.com/api/v4","AllowSignup":true,"Name":"","TlsClientCert":"","TlsClientKey":"","TlsClientCa":"","TlsSkipVerify":false,"TeamIds":[],"AllowedOrganizations":[],"AllowedGroups":null}]}
mock_1          | {"lv":"info","ts":1646783004,"msg":"AuthInit","lname":"default.log","step":"ParseAppAndSubURL","appURL":"http://localhost:19001/","appSubURL":""}
mock_1          | {"lv":"info","ts":1646783004,"msg":"init register job","lname":"ego.sys","comp":"task.ejob","name":"install"}
mock_1          | {"lv":"info","ts":1646783004,"msg":"start ejob","lname":"ego.sys","comp":"task.ejob","name":"install","tid":"598b52039d7bd5c5d6b8511df0d9f43d"}
mock_1          | {"lv":"info","ts":1646783004,"msg":"start  ejob","lname":"ego.sys","comp":"task.ejob","name":"install","tid":"598b52039d7bd5c5d6b8511df0d9f43d","cost":0.014}
mock_1          | {"lv":"error","ts":1646783004,"msg":"access","lname":"ego.sys","comp":"component.egorm","compName":"mysql.default","addr":"mysql:13306","method":"gorm:create","name":"mocro.mogo_base_instance","cost":2.239,"tid":"","event":"error","error":"Error 1062: Duplicate entry 'ch-default-ch' for key 'uix_datasource_name'"}
mock_1          | 
mock_1          | 2022/03/08 23:43:24 /mogo/api/pkg/model/db/instance.go:65 Error 1062: Duplicate entry 'ch-default-ch' for key 'uix_datasource_name'
mock_1          | [9.664ms] [rows:0] INSERT INTO `mogo_base_instance` (`datasource`,`name`,`dsn`,`rule_store_type`,`cluster_id`,`file_path`,`namespace`,`configmap`,`prometheus_target`,`ctime`,`utime`,`dtime`) VALUES ('ch','default-ch','tcp://clickhouse:9000?username=root&password=shimo&read_timeout=10&writer_timeou=10&debug=true',0,0,'','','','',1646783004,1646783004,'0')
mock_1          | panic: 
mock_1          |     msg: start up error: Error 1062: Duplicate entry 'ch-default-ch' for key 'uix_datasource_name'
mock_1          |     loc: /mogo/api/main.go:27
mock_1          | {"lv":"error","ts":1646783004,"msg":"create release error","lname":"default.log","error":"Error 1062: Duplicate entry 'ch-default-ch' for key 'uix_datasource_name'"}
mock_1          | {"lv":"error","ts":1646783004,"msg":"insert to index fail","lname":"default.log","error":"Error 1062: Duplicate entry 'ch-default-ch' for key 'uix_datasource_name'"}
mock_1          | {"lv":"panic","ts":1646783004,"msg":"start up error: Error 1062: Duplicate entry 'ch-default-ch' for key 'uix_datasource_name'","lname":"default.log","stack":"github.com/gotomicro/ego/core/elog.(*Component).Panic\n\t/go/pkg/mod/github.com/gotomicro/[email protected]/core/elog/component.go:262\ngithub.com/gotomicro/ego/core/elog.Panic\n\t/go/pkg/mod/github.com/gotomicro/[email protected]/core/elog/elog_api.go:42\nmain.main\n\t/mogo/api/main.go:27\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:255"}
mock_1          | panic: start up error: Error 1062: Duplicate entry 'ch-default-ch' for key 'uix_datasource_name'
mock_1          | 
mock_1          | goroutine 1 [running]:
mock_1          | go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc0003de000, {0x0, 0x0, 0x0})
mock_1          | 	/go/pkg/mod/go.uber.org/[email protected]/zapcore/entry.go:232 +0x446
mock_1          | go.uber.org/zap.(*Logger).Panic(0xc0000caba0, {0xc0000caba0, 0x0}, {0x0, 0x0, 0x0})
mock_1          | 	/go/pkg/mod/go.uber.org/[email protected]/logger.go:230 +0x59
mock_1          | github.com/gotomicro/ego/core/elog.(*Component).Panic(0xc0004b8e80, {0xc0000caba0, 0x59}, {0x0, 0x0, 0x0})
mock_1          | 	/go/pkg/mod/github.com/gotomicro/[email protected]/core/elog/component.go:262 +0xd0
mock_1          | github.com/gotomicro/ego/core/elog.Panic(...)
mock_1          | 	/go/pkg/mod/github.com/gotomicro/[email protected]/core/elog/elog_api.go:42
mock_1          | main.main()
mock_1          | 	/mogo/api/main.go:27 +0x178
mysql_1         | 2022-03-08T23:54:53.792590Z 34 [Note] Aborted connection 34 to db: 'mocro' user: 'root' host: '172.21.0.14' (Got an error reading communication packets)
mysql_1         | 2022-03-08T23:54:53.793707Z 35 [Note] Aborted connection 35 to db: 'mocro' user: 'root' host: '172.21.0.14' (Got an error reading communication packets)
mogo_mock_1 exited with code 2
fluent-bit_1    | [2022/03/08 23:54:58] [debug] [input:tail:tail.0] scanning path /var/log/containers/access.log
fluent-bit_1    | [2022/03/08 23:54:58] [debug] [input:tail:tail.0] scan_blog add(): dismissed: /var/log/containers/access.log, inode 461373526
fluent-bit_1    | [2022/03/08 23:54:58] [debug] [input:tail:tail.0] 0 new files found on path '/var/log/containers/access.log'

多个bug与需求

  • 需求
    • 新建日志库时间解析字段时支持datetime64类型
    • 希望可以支持全部clickhouse的sql查询语句
    • 希望可以支持多个数据库实例的相同库的表进行合并展示,而不是需要选择库后才能看到下面的表.
    • 希望在增加查询条件里支持link,in这样的查询符
    • 希望完善用户权限管理以及ldap账号登陆,以及按需开通功能,没有开通的功能不需要页面上展示
  • bug
    • 表名带有中横杠时不能添加,需要表名要以 `表名` 这种格式包起来,例如 select * from app_log.`abc-test` limit 1;
    • datetime时间字段,查询失败. 原因是时间字段的原始值被格式化了,例如 ck_time='2022-04-08 9:00:00' 被格式化成了 ck_time='2022-04-08T9:00:00+08:00'
    • datetime64时间字段,查询失败,原因是datetime64类型查询语句为: timestamp = toDateTime64('2022-04-07 21:29:29.185', 3) 而不是 timestamp = '2022-04-07 21:29:29.185'

关于日志存储到ck的问题咨询

我看了ck中nginx的log单独存了一个字段_log_,对比了其他类似的产品发现log是json化后存不同字段的。
目前存一个字段是基于因素什么考虑的?谢谢

查询25g数据表,系统直接挂了

测试系统查询性能时,查询一个表25g数据
message like '%Type%'
点击查询 查不到数据
再次访问页面时 页面无法访问
查看服务器 进程不在了。
日志:

2022/03/24 17:47:13 code: 241, message: Memory limit (total) exceeded: would use 27.57 GiB (attempt to allocate chunk of 5769656 bytes), maximum: 27.56 GiB: (while reading column message): (while reading from part /clickhouse/data/data/store/abc/abcff6a4-e6ed-4ce0-abcf-f6a4e6ed1ce0/1648112400_62943_65222_10/ from mark 5029 with max_rows_to_read = 8192): While executing MergeTreeThread

【求助】clickhouse数据库中时间是date格式,按时间筛选会报错

我计划用clickvisual查看clickhouse中的日志信息,但是引入日志表后,按时间筛选会报错。版本是v0.3.0-rc3。
Code: 386. DB::Exception: There is no supertype for types DateTime64(3), UInt32 because some of them are Date/Date32/DateTime/DateTime64 and some of them are not.
我怀疑是因为clickvisual自动生成的sql语句是将时间换成int类型,但是clickhouse不支持时间格式和int类型的直接比较。

不知道是不是我哪里配置有问题。
1653562421(1)
1653562460(1)

Mac m1 使用docker-compose部署失败

用了docker-compose.m1.yml这个文件,启动之后报错如下:

$ docker-compose up -d
Traceback (most recent call last):
  File "urllib3/connectionpool.py", line 670, in urlopen
  File "urllib3/connectionpool.py", line 392, in _make_request
  File "http/client.py", line 1255, in request
  File "http/client.py", line 1301, in _send_request
  File "http/client.py", line 1250, in endheaders
  File "http/client.py", line 1010, in _send_output
  File "http/client.py", line 950, in send
  File "docker/transport/unixconn.py", line 43, in connect
FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "requests/adapters.py", line 439, in send
  File "urllib3/connectionpool.py", line 726, in urlopen
  File "urllib3/util/retry.py", line 410, in increment
  File "urllib3/packages/six.py", line 734, in reraise
  File "urllib3/connectionpool.py", line 670, in urlopen
  File "urllib3/connectionpool.py", line 392, in _make_request
  File "http/client.py", line 1255, in request
  File "http/client.py", line 1301, in _send_request
  File "http/client.py", line 1250, in endheaders
  File "http/client.py", line 1010, in _send_output
  File "http/client.py", line 950, in send
  File "docker/transport/unixconn.py", line 43, in connect
urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "docker/api/client.py", line 214, in _retrieve_server_version
  File "docker/api/daemon.py", line 181, in version
  File "docker/utils/decorators.py", line 46, in inner
  File "docker/api/client.py", line 237, in _get
  File "requests/sessions.py", line 543, in get
  File "requests/sessions.py", line 530, in request
  File "requests/sessions.py", line 643, in send
  File "requests/adapters.py", line 498, in send
requests.exceptions.ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "docker-compose", line 3, in <module>
  File "compose/cli/main.py", line 81, in main
  File "compose/cli/main.py", line 200, in perform_command
  File "compose/cli/command.py", line 60, in project_from_options
  File "compose/cli/command.py", line 152, in get_project
  File "compose/cli/docker_client.py", line 41, in get_client
  File "compose/cli/docker_client.py", line 170, in docker_client
  File "docker/api/client.py", line 197, in __init__
  File "docker/api/client.py", line 221, in _retrieve_server_version
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
[4947] Failed to execute script docker-compose

是否支持 原样显示日志

功能描述

在可视化页面,可以看到日志分析的一些,还有日志本身的信息。这个块的展示是否支持按照日志原格式显示。类似与tail这种样式

先状态

image

期望状态

image

直接使用docker-compose 快速启动失败

panic:
msg: Casbin NewEnforcer panic
loc: /clickvisual/api/internal/service/permission/pmsplugin/pms.go:37
error: open : no such file or directory
{"lv":"panic","ts":1652324394,"msg":"Casbin NewEnforcer panic","lname":"default.log","error":"open : no such file or directory","stack":"github.com/gotomicro/ego/core/elog.(*Component).Panic\n\t/go/pkg/mod/github.com/gotomicro/[email protected]/core/elog/component.go:262\ngithub.com/gotomicro/ego/core/elog.Panic\n\t/go/pkg/mod/github.com/gotomicro/[email protected]/core/elog/elog_api.go:42\ngithub.com/clickvisual/clickvisual/api/internal/service/permission/pmsplugin.Invoker\n\t/clickvisual/api/internal/service/permission/pmsplugin/pms.go:37\ngithub.com/clickvisual/clickvisual/api/internal/service/permission.InitManager\n\t/clickvisual/api/internal/service/permission/pms.go:23\ngithub.com/clickvisual/clickvisual/api/internal/service.Init\n\t/clickvisual/api/internal/service/init.go:34\ngithub.com/gotomicro/ego.runSerialFuncReturnError\n\t/go/pkg/mod/github.com/gotomicro/[email protected]/ego_function.go:263\ngithub.com/gotomicro/ego.(*Ego).Invoker\n\t/go/pkg/mod/github.com/gotomicro/[email protected]/ego.go:133\nmain.main\n\t/clickvisual/api/main.go:16\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:255"}
panic: Casbin NewEnforcer panic

goroutine 1 [running]:
go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc00066e600, {0xc0004ae1c0, 0x1, 0x1})
/go/pkg/mod/go.uber.org/[email protected]/zapcore/entry.go:232 +0x446
go.uber.org/zap.(*Logger).Panic(0x2030b44, {0x2030b44, 0xc0004ae1c0}, {0xc0004ae1c0, 0x1, 0x1})
/go/pkg/mod/go.uber.org/[email protected]/logger.go:230 +0x59
github.com/gotomicro/ego/core/elog.(*Component).Panic(0xc0001ecd40, {0x2030b44, 0x18}, {0xc0004ae1c0, 0x1, 0x1})
/go/pkg/mod/github.com/gotomicro/[email protected]/core/elog/component.go:262 +0xd0
github.com/gotomicro/ego/core/elog.Panic(...)
/go/pkg/mod/github.com/gotomicro/[email protected]/core/elog/elog_api.go:42
github.com/clickvisual/clickvisual/api/internal/service/permission/pmsplugin.Invoker()
/clickvisual/api/internal/service/permission/pmsplugin/pms.go:37 +0x365
github.com/clickvisual/clickvisual/api/internal/service/permission.InitManager(...)
/clickvisual/api/internal/service/permission/pms.go:23
github.com/clickvisual/clickvisual/api/internal/service.Init()
/clickvisual/api/internal/service/init.go:34 +0x1b9
github.com/gotomicro/ego.runSerialFuncReturnError({0xc0006ae640, 0x2, 0x3f448e8})
/go/pkg/mod/github.com/gotomicro/[email protected]/ego_function.go:263 +0x46
github.com/gotomicro/ego.(*Ego).Invoker(0xc0006ba3c0, {0xc0002f7f40, 0x2, 0x60})
/go/pkg/mod/github.com/gotomicro/[email protected]/ego.go:133 +0x165
main.main()
/clickvisual/api/main.go:16 +0x65

没有看到相关的设置问题

import mdp/pkg/model/db

api/internal/service/event/user.go 这个文件 import mdp/pkg/model/db
where mdp/pkg/model/db
换成api/pkg/model/db 这个有过不去,userEvent.UID = opUser.Uid
代码是全的?

提个小小的建议

我看你把英文文档优先放在了中文文档前面,如果是国际化的话,那么代码里的注释尽量用英文是不是更好一点?

如果主要考虑中文用户,中文文档可以优先放在英文文档前面是不是会好一点?

cannot create directory '/bitnami/kafka/config': Permission denied

docker-compose up fail

kafka 15:23:22.71
kafka 15:23:22.71 Welcome to the Bitnami kafka container
kafka 15:23:22.71 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-kafka
kafka 15:23:22.71 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-kafka/issues
kafka 15:23:22.71
kafka 15:23:22.72 INFO  ==> ** Starting Kafka setup **
kafka 15:23:22.81 WARN  ==> You set the environment variable ALLOW_PLAINTEXT_LISTENER=yes. For safety reasons, do not use this flag in a production environment.
mkdir: cannot create directory '/bitnami/kafka/config': Permission denied

Raw log trigger conditions and alarms

hi team

I followed clickvisual and did a simple test and trial

In practical scenarios, fuzzy matching usually cannot filter logs accurately, and will cause a lot of spam, which will make us miss important content

For example, those keywords contained in the log in the last minute need to be paid attention to
The flexible matching method requires a certain SQL statement to describe the trigger condition

Trigger conditions in the UI interface should support more flexible SQL, not just option

for example

The error log line for the error keyword contains identify Sku Stock Type error but not stockout.create and entryorder.update and [INFO]

as follows

error and "identify Sku Stock Type error" not "stockout.create" not "entryorder.update" not "[INFO]"
  1. Support raw log fuzzy matching trigger
  2. Support original log exact match and multi-condition nested triggers
  3. Support original log trigger suppression function
    --> In the same matching entry, only one matching entry will be sent and the number will be counted

what do you think
thanks

queryEncode会将请求参数中带“and“的条件截断

// api/internal/service/inquiry/core.go
func queryEncode(in string) ([]queryItem, error) {
	res := make([]queryItem, 0)
	for _, a := range strings.Split(in, "and") {
		for _, op := range queryOperatorArr {
			if err := queryEncodeOperation(a, op, &res); err != nil {
				return nil, err
			}
		}
	}
	elog.Debug("queryEncode", elog.Any("step", "finish"), elog.Any("items", res))
	return res, nil
}

如果参数中包含"and"关键字,将会被截断
image
前端报错:
image

日志内容显示错误:2022-05-27T11:48:04.782+0800: 604360.660: [GC pause (G1 Evacuation Pause)

image

image

所有日志的msg内容页面上都显示为
`
2022-05-27T11:48:04.782+0800: 604360.660: [GC pause (G1 Evacuation Pause) (young)
Desired survivor size 301989888 bytes, new threshold 1 (max 1)

  • age 1: 238254864 bytes, 238254864 total
    604360.660: [G1Ergonomics (CSet Construction) start choosing CSet, _pending_cards: 23914, predicted base time: 13.34 ms, remaining time: 36.66 ms, target pause time: 50.00 ms]
    604360.660: [G1Ergonomics (CSet Construction) add young regions to CSet, eden: 122 regions, survivors: 17 regions, predicted young region time: 37.16 ms]
    604360.660: [G1Ergonomics (CSet Construction) finish choosing CSet, eden: 122 regions, survivors: 17 regions, old: 0 regions, predicted pause time: 50.50 ms, target pause time: 50.00 ms]
    , 0.0696159 secs]
    [Parallel Time: 65.1 ms, GC Workers: 16]
    [GC Worker Start (ms): Min: 604360660.1, Avg: 604360660.2, Max: 604360660.2, Diff: 0.2]
    [Ext Root Scanning (ms): Min: 1.6, Avg: 1.8, Max: 2.8, Diff: 1.2, Sum: 28.8]
    [Update RS (ms): Min: 3.6, Avg: 4.2, Max: 5.0, Diff: 1.5, Sum: 67.5]
    [Processed Buffers: Min: 4, Avg: 9.6, Max: 19, Diff: 15, Sum: 154]
    [Scan RS (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.4]
    [Code Root Scanning (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
    [Object Copy (ms): Min: 57.5, Avg: 58.4, Max: 59.0, Diff: 1.5, Sum: 934.8]
    [Termination (ms): Min: 0.0, Avg: 0.4, Max: 0.8, Diff: 0.8, Sum: 5.8]
    [Termination Attempts: Min: 1, Avg: 1.0, Max: 1, Diff: 0, Sum: 16]
    [GC Worker Other (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.6]
    [GC Worker Total (ms): Min: 64.8, Avg: 64.9, Max: 65.0, Diff: 0.2, Sum: 1037.9]
    [GC Worker End (ms): Min: 604360725.0, Avg: 604360725.0, Max: 604360725.1, Diff: 0.1]
    [Code Root Fixup: 0.0 ms]
    [Code Root Purge: 0.0 ms]
    [Clear CT: 0.5 ms]
    [Other: 4.0 ms]
    [Choose CSet: 0.0 ms]
    [Ref Proc: 2.3 ms]
    [Ref Enq: 0.2 ms]
    [Redirty Cards: 0.2 ms]
    [Humongous Regis
    `

添加clickhouse实例失败

你好,在添加clickhouse的时候失败了,提示【 {"data":{"code":1,"msg":"DNS configuration exception, database connection failure: could not load time location: unknown time zone Asia/Shanghai","data":null}} 】 ,这个是什么原因?

mogo尝试配置的数据库类型有mysql5.6、mysql5.7、tidb5.3;
mogo运行的方式有二进制、docker、k8s 。

Loading k8s configmap error

添加k8s集群后能获取Configmaps但无法在面板展示内容,点击从k8s同步 error {"data":{"code":1,"msg":"sql: Scan error on column index 12, name "lock_at": converting driver.Value type time.Time ("0001-01-01 00:00:00 +0000 UTC") to a int64: invalid syntax","data":"sql: Scan error on column index 12, name "lock_at": converting driver.Value type time.Time ("0001-01-01 00:00:00 +0000 UTC") to a int64: invalid syntax"}}

windows下启动运行页面报404

embed包中的,文件夹分隔符采用正斜杠/,即使是windows系统也采用这个模式。
image

image

在windows下运行,路径会拼接以为"\"分隔符的路径形式,文件将找不到。

image

类似的issue: golang/go#45230

feat: 希望支持clickhouse导入阿里云日志系统

1 clickhouse完全开源,可以自行搭建节约成本。 但是没有一个好用的可视化界面,查询语法的能力较阿里云日志也相对逊色一些。
对于有特殊需求的日志数据,是否可以通过一键导入到阿里云日志系统,方便分析使用。 对于没有太多价值的日志,用clickhouse查查就可以了。

趋势图步长建议与图表展示

  • 优化

    • 趋势图增加步长与点数的设置. 比如设置最小步长为1分钟最大点位为30,那么15分钟的数据就显示15个趋势点位(一分钟一个点位),如果30分钟的数据就显示30个点位(一分钟一个点位),如果是1个小时的数据就显示最大30个点位(两分钟一个点位)
  • 新功能

    • 增加以表的形式显示数据,并增加切换按钮用户可自行选择以列的形式展示数据或者以表的形式展示数据.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.