dennyzhang / monitor-docker-slack Goto Github PK

View Code? Open in Web Editor NEW

73.0 8.0 44.0 208 KB

:alarm_clock: Get Slack Notifications, When Containers Run Into Issues

Home Page: https://www.dennyzhang.com/docker_monitor

Python 78.42% Shell 3.75% Dockerfile 17.84%

python docker slack monitoring devops containers

monitor-docker-slack's Introduction

File me tickets or star the repo

Introduction

Get Slack Notifications, When Containers Run Into Issues

General Idea

Start a container in the target docker host.
This container will query status for all containers.

curl -XGET --unix-socket /var/run/docker.sock http://localhost/containers/json

Send slack notifications, we get matched of "unhealthy"

How To Use: Plain Container

Specify slack credentials via env

export SLACK_CHANNEL="#XXX"
export SLACK_USERNAME="XXX"
export SLACK_TOKEN="xoxp-XXX-XXX-XXX-XXXXXXXX"
export MSG_PREFIX="Monitoring On XX.XX.XX.XX"

Start container to check

container_name="monitor-docker-slack"
# Stop and delete existing container
docker stop $container_name; docker rm "$container_name"

# Start container to monitor docker healthcheck status
docker run -v /var/run/docker.sock:/var/run/docker.sock \
   -t -d -h $container_name --name $container_name \
   -e SLACK_CHANNEL="$SLACK_CHANNEL" -e SLACK_USERNAME="$SLACK_USERNAME" \
   -e SLACK_TOKEN="$SLACK_TOKEN" -e MSG_PREFIX="$MSG_PREFIX" \
   -e WHITE_LIST="$WHITE_LIST" --restart=always \
   denny/monitor-docker-slack:latest

# Check status
docker logs "$container_name"

How To Use: Docker-compose

version: '2'
services:
  monitor-docker-slack:
    container_name: monitor-docker-slack
    image: denny/monitor-docker-slack:latest
    volumes:
     - /var/run/docker.sock:/var/run/docker.sock
    environment:
      SLACK_CHANNEL: "#XXX"
      SLACK_USERNAME: "XXX"
      SLACK_TOKEN: "xoxp-XXX-XXX-XXX-XXXXXXXX"
      MSG_PREFIX: "Monitoring On XX.XX.XX.XX"
    restart: always

More customization

Add message prefix for the slack notification

export MSG_PREFIX="Docker Env in Denny's env"

Skip checking certain containers by customizing WHITE_LIST env.

export MSG_PREFIX="Docker Env in Denny's env"
export WHITE_LIST="nodeexporter,ngin.*"

Code is licensed under MIT License.

monitor-docker-slack's People

Contributors

Stargazers

Watchers

Forkers

rajeshksingh15 bharu456 aiminickwong gthoma17 hemanth7787 stenpiren naitikshah bansalniranjan petekaik michaelbukachi nishant23 sohussain indrajithi peppersec wazirsoft rajeshv23 svelix jannatulhoque modulexcite rafahsena yongwang kanopai chohonest thepfmind dimiactor icelander cinco flaviocarmo poohsen techsd olatolu ebsud89 lapnd rajkumar-dev143 ugummall dhruv-sachdev1313 mstrychalski datalabfabriek

monitor-docker-slack's Issues

Use webhook instead of legacy token

Tokens are no longer used with Slack, it would be nice to be able to use a webhook url instead

the monitor docker did not send the notification to my channel in slack.

I have recevied none of the notification from the docker monitor and I think I set everything right to the requirement. No error occurs when running the monitor-docker-slack.sh.

Problem with SSL Certificates

I have removed all containers except "test" one.
Then I ran python script "monitor-docker-slack.py" and stopped container "test"
Afterr 300 seconds I see this. Seems like script cannot send message to slack.com:443

(app-root) bash-4.2# /monitor-docker-slack.sh
+ python /monitor-docker-slack.py --check_interval 300 --slack_token XXXXXXXXX/XXXXXXXXX/XXXXXXXXXXXXXXXXXXXXXXXX --whitelist '' --slack_channel unixteam --msg_prefix 'Monitoring on ncatlnxv17'
OK: Monitoring on ncatlnxv17
OK: detect no stopped or unhealthy containers
ERROR: Monitoring on ncatlnxv17
Detected Stopped Containers: 
['/test']: Exited (0) 4 minutes ago


Traceback (most recent call last):
  File "/opt/app-root/lib/python3.6/site-packages/urllib3/connectionpool.py", line 595, in urlopen
    self._prepare_proxy(conn)
  File "/opt/app-root/lib/python3.6/site-packages/urllib3/connectionpool.py", line 816, in _prepare_proxy
    conn.connect()
  File "/opt/app-root/lib/python3.6/site-packages/urllib3/connection.py", line 326, in connect
    ssl_context=context)
  File "/opt/app-root/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 329, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 814, in __init__
    self.do_handshake()
  File "/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 1068, in do_handshake
    self._sslobj.do_handshake()
  File "/opt/rh/rh-python36/root/usr/lib64/python3.6/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/app-root/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/opt/app-root/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/opt/app-root/lib/python3.6/site-packages/urllib3/util/retry.py", line 388, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='slack.com', port=443): Max retries exceeded with url: /api/chat.postMessage (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)'),))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/monitor-docker-slack.py", line 151, in <module>
    channel = slack_channel, text = err_msg)
  File "/opt/app-root/lib/python3.6/site-packages/slackclient/client.py", line 83, in api_call
    result = json.loads(self.server.api_call(method, timeout=timeout, **kwargs))
  File "/opt/app-root/lib/python3.6/site-packages/slackclient/server.py", line 220, in api_call
    return self.api_requester.do(self.token, method, kwargs, timeout=timeout).text
  File "/opt/app-root/lib/python3.6/site-packages/slackclient/slackrequest.py", line 85, in do
    proxies=self.proxies)
  File "/opt/app-root/lib/python3.6/site-packages/requests/api.py", line 112, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/opt/app-root/lib/python3.6/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/opt/app-root/lib/python3.6/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/opt/app-root/lib/python3.6/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/opt/app-root/lib/python3.6/site-packages/requests/adapters.py", line 506, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='slack.com', port=443): Max retries exceeded with url: /api/chat.postMessage (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)'),))

Missing one important config property Slack Workspace

My user name and channel name is not enough to properly target a message. Groups of users are organized in workspaces in Slack and I can have the same channel in multiple Workspaces. The alert was visible in docker container log but never made it to Slack when I set the username and auth token. No error message is visible in docker logs .

BIG SECURITY Problem

Hello,

This image is based on a very insecure python image.
You can see the scan here:
https://anchore.io/image/dockerhub/denny%2Fmonitor-docker-slack%3Alatest

14009 security problems, you should use a differente base image.

This is a good work, but without worries about security.

Is there any way I can send notification to email?

First of all, I thank you for this project. I am struggling to send notifications to email id whenever any containers restart.
If you could give me some pointers, I can try to make this work and may contribute it here as well.