Code Monkey home page Code Monkey logo

cronmon's Introduction

cronmon

cronmon 是一个计划任务(定时任务)监控系统,可以对循环执行的程序和脚本进行监控告警,当其未按照预期执行时,发送邮件到对应邮箱进行通知。 同时可以将监控任务划分到不同业务下面,每个业务可以分配不同的通知人,建立业务、通知人和监控任务的多层级关系。

通过以一定的间隔发送HTTPS请求到特定的URL实现监控。如果URL未按时接受到请求,对应的业务通知人则会收到告警。 你可以监控你的数据库备份和安全扫描等关键性任务。

https://travis-ci.org/bruceye777/cronmon.svg?branch=master https://coveralls.io/repos/github/bruceye777/cronmon/badge.svg?branch=master

安装

从github下载安装

$ git clone [email protected]:bruceye777/cronmon.git
$ python setup.py install  # 如果无需安装到site-packages目录,此步可省略。

初始化

解压缩/安装之后,需要进行初始化工作,包括全局配置修改、系统表初始化和样例数据插入。

$ vim cronmon/conf/config.py  # 全局配置修改

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/globalConfig.png

$ python migrate.py init  # 系统表初始化,初始用户名和密码均为cronadmin
$ python migrate.py sample  # 样例数据插入

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/init.png

启停程序

$ uwsgi cronmon.ini&  # 启动程序
$ uwsgi --stop /var/run/uwsgi_cronmon.pid  # 停止程序

如果将cronmon安装成服务(使用production/cronmon文件)

$ service cronmon start  # 启动程序
$ service cronmon stop  # 停止程序

使用

  • 登录;

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/login.png

  • 登录之后,看到的首页;

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/home.png

  • 业务管理相关操作;

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/businesslist.png

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/businessedit.png

  • 通知人管理相关操作;

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/notifylist.png

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/notifyedit.png

  • 业务通知人管理相关操作;

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/businessnotifylist.png

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/businessnotifyedit.png

  • 任务管理相关操作;

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/tasklist.png

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/taskedit.png

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/taskloglist.png

  • 系统管理相关操作;

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/permlist.png

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/permedit.png

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/passwordreset.png

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/permbizedit.png

  • 用户相关操作;

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/user.png

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/passwordchange.png

  • 业务管理员(普通用户)操作界面,业务通知人和任务可以查询编辑,通知人仅限查询;

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/normalUser.png

API调用

获取所有监控任务

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/apiTasksAll.png

按指定关键字获取任务(业务名、任务名和任务URL)

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/apiTasksTaskname.png

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/apiTasksBizname.png

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/apiTasksUrl.png

公共参数包括page(显示第几页)和length(每页显示多少记录)

如何使用监控链接

在不同情况下调用任务监控URL的写法

  • crontab写法
# system status check
20 * * * * cd /path/to/systemStatusCheck && ./systemStatusCheck.sh > systemStatusCheck.sh.cron.log 2>&1 && curl -kfsS --retry 3 --connect-timeout 10 --ipv4 https://cronmon.yoursite.io/api/monlink/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx >> systemStatusCheck.sh.cron.log 2>&1
  • bash写法
$ curl -kfsS --retry 3 --connect-timeout 10 --ipv4 https://cronmon.yoursite.io/api/monlink/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
  • python写法
>>> import requests
>>> requests.get('https://cronmon.yoursite.io/api/monlink/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx')

生产环境

  • 服务启停

以CentOS为例,首先编辑相关变量

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/serviceManageConfig.png

然后将启动脚本(production/init.d/cronmon)放入/etc/init.d目录

$ service cronmon help
Usage: cronmon {start|stop|restart|reload|status|help}
$ chkconfig --level 35 cronmon on  # 加入开机启动
  • 日志分区

针对日志表数据,为了方便管理和提高效率,使用mysql分区,首先导入存储过程创建sql文件(production/cronmonPartition.sql), 脚本首先修改表结构,添加了复合主键(id+create_datetime),然后创建了所需的5个存储过程,脚本基于zabbix分区创建脚本进行修改 (链接https://www.zabbix.org/wiki/Docs/howto/mysql_partition#MySQL_Database_Partitioning),执行脚本后结果如下:

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/mysqlPartitionTableStructure.png

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/mysqlPartitionProcedures.png

对应的shell脚本

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/mysqlPartitionShellScript.png

对应的crontab配置

# cronmon mysql partition
15 0 * * * cd /path/to/cronmonMysqlPartition && ./cronmonMysqlPartition.sh > cronmonMysqlPartition.sh.cron.log 2>&1 && curl -kfsS --retry 3 --connect-timeout 10 --ipv4 https://cronmon.yoursite.io/api/monlink/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx >> cronmonMysqlPartition.sh.cron.log 2>&1
  • WebService配置

以nginx为例

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/nginxConfig.png

  • 日志轮转

以logrotate为例

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/logRotate.png

数据库结构变更

如果要新增表或者修改表结构,通过如下方式进行:

$ vim migrate.py  # 修改表结构定义文件

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/createNewTable.png

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/alterCurrentTable.png

压力测试

  • 配置调整

uwsgi相关配置

[uwsgi]
listen=10240
processes=8
gevent=100

系统相关配置

# echo 'net.core.somaxconn=4096' >> /etc/sysctl.conf
  • 压测表现

使用ab对监控链接进行压力测试,10000并发无错误(目前只测试到10000并发,未测试更高并发),客户端和服务端网络延迟在11ms左右:

$ ab -n 50000 -c 10000 https://cronmon.yoursite.com/api/monlink/ba60afc4-f162-11e8-a56a-001b7872e686

https://raw.githubusercontent.com/bruceye777/cronmon/master/docs/images/abStressTest.png

贡献

如果您想贡献代码,只需fork本仓库,然后push您的更改并发送pull请求。

获取帮助

如果您有任何问题或建议,请在此仓库中打开一个issue,我会尽力提供帮助。

cronmon's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cronmon's Issues

English

Hello,
Can you please provide cronmon and its documentation in English version? That would be great. Thank you.

windows调试?

Windows环境用不了uwsgi。。。

很多文件编码也无法在windows上读取
例如:
conf/log.conf

希望作者可以提供windows版本以供调试

There was a problem inserting the sample data

Traceback (most recent call last):
File "migrate.py", line 205, in
manager.run()
File "/usr/local/python3/lib/python3.6/site-packages/flask_script/init.py", line 417, in run
result = self.handle(argv[0], argv[1:])
File "/usr/local/python3/lib/python3.6/site-packages/flask_script/init.py", line 386, in handle
res = handle(*args, **config)
File "/usr/local/python3/lib/python3.6/site-packages/flask_script/commands.py", line 216, in call
return self.run(*args, **kwargs)
File "migrate.py", line 159, in sample
Business.insert_many(business_data, fields=business_fields).execute()
File "/usr/local/python3/lib/python3.6/site-packages/peewee.py", line 1625, in inner
return method(self, database, *args, **kwargs)
File "/usr/local/python3/lib/python3.6/site-packages/peewee.py", line 1696, in execute
return self._execute(database)
File "/usr/local/python3/lib/python3.6/site-packages/peewee.py", line 2358, in _execute
return super(Insert, self)._execute(database)
File "/usr/local/python3/lib/python3.6/site-packages/peewee.py", line 2121, in _execute
cursor = database.execute(self)
File "/usr/local/python3/lib/python3.6/site-packages/peewee.py", line 2727, in execute
return self.execute_sql(sql, params, commit=commit)
File "/root/cronmon/cronmon/models.py", line 21, in execute_sql
cursor = super(RetryOperationalError, self).execute_sql(sql, params, commit)
File "/usr/local/python3/lib/python3.6/site-packages/peewee.py", line 2721, in execute_sql
self.commit()
File "/usr/local/python3/lib/python3.6/site-packages/peewee.py", line 2512, in exit
reraise(new_type, new_type(*exc_args), traceback)
File "/usr/local/python3/lib/python3.6/site-packages/peewee.py", line 186, in reraise
raise value.with_traceback(tb)
File "/usr/local/python3/lib/python3.6/site-packages/peewee.py", line 2714, in execute_sql
cursor.execute(sql, params or ())
File "/usr/local/python3/lib/python3.6/site-packages/pymysql/cursors.py", line 170, in execute
result = self._query(query)
File "/usr/local/python3/lib/python3.6/site-packages/pymysql/cursors.py", line 328, in _query
conn.query(q)
File "/usr/local/python3/lib/python3.6/site-packages/pymysql/connections.py", line 517, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/usr/local/python3/lib/python3.6/site-packages/pymysql/connections.py", line 732, in _read_query_result
result.read()
File "/usr/local/python3/lib/python3.6/site-packages/pymysql/connections.py", line 1075, in read
first_packet = self.connection._read_packet()
File "/usr/local/python3/lib/python3.6/site-packages/pymysql/connections.py", line 684, in _read_packet
packet.check_error()
File "/usr/local/python3/lib/python3.6/site-packages/pymysql/protocol.py", line 220, in check_error
err.raise_mysql_exception(self._data)
File "/usr/local/python3/lib/python3.6/site-packages/pymysql/err.py", line 109, in raise_mysql_exception
raise errorclass(errno, errval)
peewee.IntegrityError: (1062, "Duplicate entry '????' for key 'business_business_name'")

初始化提示找不到config模块

你好,centos7 python 2.7.5 setup之后 初始化报错了:

[root@locahost cronmon]# python migrate.py init
Traceback (most recent call last):
  File "migrate.py", line 6, in
    from cronmon import create_app
  File "/usr/local/cronmon/cronmon/init.py", line 10, in
    from cronmon.conf.config import config
ImportError: No module named conf.config
git上的issues不知道为什么 /usr/local/cronmon/cronmon/init.py 这个文件的符号“—”不见了实际上是有的。
找不到模块,请问是什么原因?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.