ireaderlab / zkdash Goto Github PK

View Code? Open in Web Editor NEW

682.0 76.0 208.0 4.81 MB

A dashboard for zookeeper and Qconf

License: Apache License 2.0

Python 5.25% JavaScript 79.78% Shell 0.02% CSS 9.29% PHP 2.04% HTML 3.61%

zookeeper qconf dashboard

zkdash's Introduction

zkdash

zkdash是一个zookeeper的管理界面，也可以作为任何基于zookeeper的配置管理项目比如：Qconf

开发规划

zkdash目前正在开发第二版本，更易用，更开发，拥有完善权限管理，支持zk，更易用。第二版已经在着手开源相关工作，会打造一个基于配置中心，服务发现，CICD，CMDB一体化的微服务管理平台。

zkdash页面展示

安装与运行

安装步骤

安装并运行mysql
安装python2.7, 版本最好 >= 2.7.6
下载zkdash git clone https://github.com/ireaderlab/zkdash.git

安装依赖项

cd zkdash
pip install -r requirements.txt

设置配置文件

根据需要修改当前目录下./conf/conf.yml中相关配置信息，配置文件详细说明见后面

同步数据库表结构首先创建数据库zkdash，并设置数据库的用户名和密码将配置文件的数据库的用户名和密码进行修改

DATABASE:
  db: 'zkdash'
  host: '192.168.1.1'   # 修改为你的数据库地址
  port: 3306   # 设置端口号
  user: 'tokyo'   # 修改用户名
  passwd: 'tokyo!'  # 修改密码

设置完成后进行初始化数据库

cd zkdash
python ./bin/syncdb.py   # 注意执行路径必须为./bin/syncdb.py

说明：数据库使用mysql，创建表结构前请先配置数据库连接信息

运行
```
cd zkdash
python init.py -port=8888
```
说明：初次运行zkdash时需要到zookeeper管理菜单下增加监控的zookeeper集群ip信息

配置文件说明

配置文件详细说明

数据库配置项(DATABASE)

db: 数据库名称
host: ip地址
port: 端口号
user: 用户名
passwd: 密码

全局配置项

USE_QCONF: 是否通过QConf获取zookeeper数据（使用该项可以提高树形展示配置信息的响应速度）

与QConf的搭配使用

反馈服务器地址：http://ip:port/api/v1/feedback（Agent同步状况查看依赖此反馈信息）

注意事项

新增节点需要先指定父节点，并且只能逐级增加
当设置使用QConf获取zookeeper数据时，zookeeper管理菜单下的zookeeper集群名称需要与QConf 客户端Agent配置文件的idc名称一致

zkdash's People

Contributors

Stargazers

Watchers

Forkers

paderma niubell caipanjin shejianmin zhanglei lmtwga cxianf kyleguo yc8332 zogwei lilirui520 lupan2015 ifa6 yikey scwanglijun haiyang1987 frandavid oopsoutofmemory wangjianbao zhaozhxcn mqwhub iyangming shineforever yourchanges fengshao0907 zhausong trentsky mtanlee sdgdsffdsfff lorgine-li assad2008 gavinljj hecran abbabb123 cyc6420 dengqiao open-party chunmeizhang915 is00hcw wudi003 amor-tsai tomzhang lotusswan xindaya wangfakang hoongyi jiajie999 duduwang pkzikao chenbk85 marqio hopeful logonmy zj8487 tangtangsara giddily wuchengjiang nirousi pylixm wangycc lue828 lviiii agostop shimaomao yaoml wuwutai jzmq nicholaskh zhaohaolin liucyao lmy2011 fnet123 shixin42 jsongo vwyuheng enyun yiduwangkai xiongxin thrbowl kerasking afghanistanyn sandy-yan macrowh fengyindiehun xiaomatech xiaocok magicleo cqyhm koone yanmaipian sunnyxd xiaodong84 bourneye huanglg cricense morganwang010 culiangfanzi kainhong cunnainiuhaohe stableship

zkdash's Issues

Traceback (most recent call last):
File "init.py", line 86, in
main()
File "init.py", line 78, in main
application = Application()
File "init.py", line 35, in init
routes = load('handler')
File "/root/zkdash/lib/utils/routes.py", line 109, in load
import(modname)
File "/root/zkdash/handler/config/znode.py", line 27, in
from service import znode as ZnodeService
File "/root/zkdash/service/znode.py", line 16, in
from lib.zyqconf import qconf_py
File "/root/zkdash/lib/zyqconf/init.py", line 12, in
import qconf_py
qconf_py.Error: inital qconf evn failed!

点击“批量增改”弹出窗口，当键值为Json格式时无法正常显示，显示为[object Object]

关于zkdash/lib/zyqconf/qconf_py.so源码

zkdash/lib/zyqconf/qconf_py.so

这个源码不会公开吗?

添加agent不能选择zk集群

集群情况: 配置了一个集群
操作系统: MacOS
浏览器 : Chrome

指定监听IP

启动的时候能指定监听IP吗

[bug]貌似未auth会出错。

D 170109 13:38:47 connection:282] Sending request(xid=1173): GetChildren(path=u'/elastic-job-example-lite-spring/springSimpleJob/servers/10.65.128.186/trigger', watcher=None)
[D 170109 13:38:47 connection:352] Received error(xid=1173) NoAuthError((), {})
[E 170109 13:38:47 base:132] Internal Error: Traceback (most recent call last):
      File "/home/jinlei1/tools/zkdash/handler/bases/base.py", line 128, in _exec
        res = self.response()
      File "/usr/local/lib/python2.7/site-packages/tornado/web.py", line 2601, in wrapper
        return method(self, *args, **kwargs)
      File "/home/jinlei1/tools/zkdash/handler/config/znode.py", line 77, in response
        ZnodeService.get_znode_tree(zoo_client, normalized_path, nodes)
      File "/home/jinlei1/tools/zkdash/service/znode.py", line 146, in get_znode_tree
        get_znode_tree(zoo_client, child_path, nodes, child_id, current_id)
      File "/home/jinlei1/tools/zkdash/service/znode.py", line 146, in get_znode_tree
        get_znode_tree(zoo_client, child_path, nodes, child_id, current_id)
      File "/home/jinlei1/tools/zkdash/service/znode.py", line 146, in get_znode_tree
        get_znode_tree(zoo_client, child_path, nodes, child_id, current_id)
      File "/home/jinlei1/tools/zkdash/service/znode.py", line 146, in get_znode_tree
        get_znode_tree(zoo_client, child_path, nodes, child_id, current_id)
      File "/home/jinlei1/tools/zkdash/service/znode.py", line 146, in get_znode_tree
        get_znode_tree(zoo_client, child_path, nodes, child_id, current_id)
      File "/home/jinlei1/tools/zkdash/service/znode.py", line 136, in get_znode_tree
        children = zoo_client.get_children(path)
      File "/usr/local/lib/python2.7/site-packages/kazoo/client.py", line 1078, in get_children
        return self.get_children_async(path, watch, include_data).get()
      File "/usr/local/lib/python2.7/site-packages/kazoo/handlers/utils.py", line 78, in get
        raise self._exception
    NoAuthError: ((), {})

树形展示配置信息优化

现在如果直接从根查起，貌似是一次性遍历所有节点出来的？
如果改成逐级展示，打开某个父节点的时候再去取子节点的信息会不会更能应对配置较多的场景？

启动报错：初始化数据库

python ./bin/syncdb.py
Traceback (most recent call last):
File "./bin/syncdb.py", line 19, in
from model.db.base import ZKDASH_DB
File "/Users/peirong.jiang/PycharmProjects/zkdash/model/db/base.py", line 12, in
from peewee import Field
ImportError: No module named peewee

第二版本啥时候发布?

说好的“更易用，更开发，拥有完善权限管理，支持zk，更易用”的第二版本呢？

zkdash 没有登录校验只能在局域网内用用了，放在公网，信息容易泄露

根据文档修改配置文件中的 Use_conf 无法获取子节点

修改了配置文件中的use_conf=True
重启zkdash 点击配置管理后台显示 node does not exists on Qconf agent:
agent 手动 get了一下还是有如上错误
请问哪里配置不对?

另: 配置完zookeeper后 agent是否只能手动配置?

能不能加上权限验证，这样的后台希望登陆后才能访问

Internal Error: Traceback (most recent call last):

"SELECT node.id, node.node_path, (COUNT(parent.node_path)-1) AS depth FROM zd_snapshot_tree AS node, zd_snapshot_tree AS parent WHERE node.cluster_name = 'test' AND parent.cluster_name = 'test' AND node.node_path like '/%%' AND node.left BETWEEN parent.left AND parent.right GROUP BY node.node_path ORDER BY node.left"

这个sql 报错了

Node does not exists on QConf Agent, path: /test

日志是这个，使用QConf，查不到数据?什么情况

'MyRetryDB' object has no attribute 'field_overrides'

[root@localhost bin]# python syncdb.py
Traceback (most recent call last):
File "syncdb.py", line 19, in
from model.db.base import ZKDASH_DB
File "/home/danneycao/open_src/zkdash/model/db/base.py", line 18, in
ZKDASH_DB = Database(**DATABASE)
File "/home/danneycao/open_src/zkdash/lib/db/database.py", line 26, in init
self.load_database()
File "/home/danneycao/open_src/zkdash/lib/db/database.py", line 32, in load_database
self.database.field_overrides.update({'enum': 'enum'}) # 增加枚举类型
AttributeError: 'MyRetryDB' object has no attribute 'field_overrides'

疑问

url 参数中如果带有特殊字符 ‘#’ 会导致url不可用，或参数缺失。

request: config/znode/view?path=/test/office/dataproxy/data#test_dataproxy_1_155&cluster_name=办公网Zookeeper
{"status": 1000, "msg": "参数错误，请检查参数后再请求! cluster_name 参数为必填参数!"}

nImportError: libpython2.7.so.1.0 什么情况？

zkdash/lib/zyqconf/init.py", line 12, in \n import qconf_py\nImportError: libpython2.7.so.1.0: cannot open shared object file: No such file or directory\n

建议

能不能把节点值: 都更改为areatext
这样配置文件是不是就能保持格式，增强可读性？

zkdash是否支持配置node权限？

请问有没有开源社区

有没有社区，qq群之类的方便大家沟通的地方？

更新配置问题

在更新配置的时候，如果配置项为文件，则无法直接上传新文件更新，仅能下载配置文件。只有在切换节点类型后，才能再次上传文件。

数据库断开连接后重连失败

错误日志:
error.log
[WARNING 2015-10-19 10:04:48,530 base.py 133] Mysql OperationalError: (2006, 'MySQL server has gone away')
zkdash.log
[INFO 2015-10-19 10:04:48,515 base.py 25] 重连数据库...
[INFO 2015-10-19 10:04:48,516 init.py 65] 302 GET /config/snapshot/index?target_tab=_config_snapshot&xsrf=2%7C17bfb130%7Cdc3a69e4a29362a6b33165d284f8b5b3%7C1444999331&=1445220289410 (36.110.17.2) 2.02ms
[DEBUG 2015-10-19 10:04:48,529 peewee.py 2865]('SELECT Count%28*%29 FROM zd_zookeeper AS t1 WHERE %28t1.deleted = %s%29', ['0'])
[WARNING 2015-10-19 10:04:48,530 base.py 133] Mysql OperationalError: (2006, 'MySQL server has gone away')

关于运行python init.py -port=8888的一个错误

File "/usr/lib/python2.7/site-packages/kazoo/hosts.py", line 3, in
from six.moves import urllib_parse
ImportError: cannot import name urllib_parse

mac下运行报错

[E 160918 14:04:51 base:132] Internal Error: Traceback (most recent call last):
File "/Users/xinglongliu/git/zkdash/handler/bases/base.py", line 128, in _exec
res = self.response()
File "/Library/Python/2.7/site-packages/tornado/web.py", line 2601, in wrapper
return method(self, _args, *_kwargs)
File "/Users/xinglongliu/git/zkdash/handler/config/znode.py", line 72, in response
self.cluster_name, normalized_path, nodes)
File "/Users/xinglongliu/git/zkdash/service/znode.py", line 152, in get_znode_tree_from_qconf
from lib.zyqconf import qconf_py
File "/Users/xinglongliu/git/zkdash/lib/zyqconf/init.py", line 12, in
import qconf_py
ImportError: dlopen(/Users/xinglongliu/git/zkdash/lib/zyqconf/qconf_py.so, 2): no suitable image found. Did find:
/Users/xinglongliu/git/zkdash/lib/zyqconf/qconf_py.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00

zookeeper断开连接后不可以重建连接

报错情况如下：
[ERROR 2015-10-20 09:26:57,325 base.py 140] Internal Error: Traceback (mos t recent call last):
File "zkdash/handler/bases/base.py", lin e 130, in _exec
res = self.response()
File "/usr/local/lib/python2.7/dist-packages/tornado/web.py", line 2601, in wrapper
return method(self, _args, *_kwargs)
File "zkdash/handler/config/agent.py", l ine 95, in response
zk_agents = zoo_client.get_children(self.agent_register_prefix)
File "/usr/local/lib/python2.7/dist-packages/kazoo/client.py", line 1078 , in get_children
return self.get_children_async(path, watch, include_data).get()
File "/usr/local/lib/python2.7/dist-packages/kazoo/handlers/utils.py", l ine 72, in get
raise self._exception
ConnectionClosedError: Connection has been closed
可能原因:
zkdash/service/zookeeper.py line 34没有判断已建立连接的状态

怎么以后台程序运行

我运行python init.py -port=8888 & 后，因为程序在运行过程中有大量日志打印，我关闭了终端，就无法访问web页面了