Code Monkey home page Code Monkey logo

joshuasmeda / thehive_sla_monitor Goto Github PK

View Code? Open in Web Editor NEW
9.0 3.0 1.0 116 KB

Query and cross-check TheHive (SIRP) alerts based on set severity statuses, and automatically perform various escalations based on your configuration. Integrates with Slack, Twilio, Flask and TheHive.

Home Page: https://thehive-project.org/

License: GNU General Public License v3.0

Python 99.12% Dockerfile 0.88%
slack slack-bot slack-api twilio twilio-sms-api twilio-voice hive thehive-project thehive4py python

thehive_sla_monitor's Introduction

contributions welcome build_python commit_activity

TheHive_SLA_Monitor (Poorman's Opsgenie)

This applications runs as a standalone script or as a service, queries TheHive (SIRP) alerts based on set severity statuses while cross checking custom set SLA limits, and SMS's or calls specified people if there is a breach or escalation necessary.

Here's an general idea of how it works: The TheHive_SLA_Monitor application queries TheHive API and will poll for any existing Hive alerts as depicted below:

carbon (3)

If a alert hits a pre-defined threshold, you will see an alert fire - how you want to be alerted is up to you!

carbon (4)

Overview:

This application connects to your Slack workspace using the Slack API. The application periodically polls the TheHive using a pre-defined API key. It grabs all alerts with a severity status of your choice (specific tiers, or all 3) and performs SLA checks on the alerts. The SLA's are up-to-you to decide on what you want them to be, but here is a general idea to get you going:

30 minutes - SMS person scheduled "on-call"
45 minutes - Call person scheduled "on-call" and echo a generic, pre-defined, message via Twilio using a Twimlet. This serves as an additional alert.
60 minutes - Escalate to seniors engineers, through phone call via Twilio

Once a alert fires, it won't be re-alert on unless it hits a new SLA tier (e.g. Moves from 30 minutes to 45 minutes).

Each alert will create a slack notice that allows you to promote to case or ignore the alert for 30 minutes from within Slack, instead of manually acknowledging the alert via the TheHive UI / API. When promoting a case, Slack will link you to the imported case for convenience. When ignoring, the alert won't re-alert at 45 / 60 minutes for 30 minutes.

A log record is generated each transactional event which allows you to track and audit events.

Configuration parameters:

The configuration.py is utilized as a central location to modify functionality and parameters related to the application. Such as, changing the severity level you wish to search TheHive for, or adjusting the SLA timers you wish to poll at.

The following explains the parameters accepted:


SLA_SETTINGS = {
     # Int: seconds to classify the age of a low severity SLA breach in the form of seconds
     # Note: Do not adjust the key name as it's used in conjunction with a switch case function within the program.
     # This nested dictionary allows you to configure whether you want to enable alerting for each of the 3 tiers provided by TheHive. Adjust the TIMER accordingly based on your SLA requirements
     # and the NOTIFICATION_METHOD. You can include a single method, or even multiple if your use-case requires so.
     # The HIGH_RISK object is necessary for any high_word_risk triggers - Do NOT Remove!

     'THEHIVE_LEVEL1': {'ENABLED': True,
                       'LOW_SEVERITY': {'TIMER': 1800, 'NOTIFICATION_METHOD': ['SLACK_API', 'TWILIO_SMS']},
                       'MEDIUM_SEVERITY': {'TIMER': 2700, 'NOTIFICATION_METHOD': ['SLACK_API', 'TWILIO_ESC']},
                       'HIGH_SEVERITY': {'TIMER': 3600, 'NOTIFICATION_METHOD': ['SLACK_API', 'TWILIO_CALL']},
                       'HIGH_RISK': {'NOTIFICATION_METHOD': ['SLACK_API', 'TWILIO_CALL']}},

     'THEHIVE_LEVEL2': {'ENABLED': True,
                       'LOW_SEVERITY': {'TIMER': 1800, 'NOTIFICATION_METHOD': ['TWILIO_SMS']},
                       'MEDIUM_SEVERITY': {'TIMER': 2700, 'NOTIFICATION_METHOD': ['SLACK_API', 'TWILIO_SMS']},
                       'HIGH_SEVERITY': {'TIMER': 3600, 'NOTIFICATION_METHOD': ['SLACK_API', 'TWILIO_CALL']},
                       'HIGH_RISK': {'NOTIFICATION_METHOD': ['SLACK_API', 'TWILIO_CALL']}},

     'THEHIVE_LEVEL3': {'ENABLED': True,
                       'LOW_SEVERITY': {'TIMER': 1800, 'NOTIFICATION_METHOD': ['SLACK_API']},
                       'MEDIUM_SEVERITY': {'TIMER': 2700, 'NOTIFICATION_METHOD': ['SLACK_API', 'TWILIO_SMS']},
                       'HIGH_SEVERITY': {'TIMER': 3600, 'NOTIFICATION_METHOD': ['TWILIO_CALL']},
                       'HIGH_RISK': {'NOTIFICATION_METHOD': ['SLACK_API', 'TWILIO_CALL']}}
}

SYSTEM_SETTINGS = {
    'HIGH_RISK_WORDS': ['CRITICAL', 'URGENT', 'FAILURE']  # List: Add custom words here that you want to be critically alerted on. These words must be included (non-case sensitive) in the Hive title or in one of the artifacts. This will immediately escalate to HIGH_SEVERITY SLA.
    'HIGH_RISK_WORDS_SEVERITY_LEVEL': 2, # Integer: Adjust the specific severity level you want high_risk_words to specifically run on. For example, if you only want high_risk_words triggers on TheHive severity 3 alerts.
    'LOOP_TIME': 120, # Integer: Adjust the amount of time in seconds the application must re-poll TheHive.
    'MAX_ALERT_DETECTION_ENABLED': True, # Boolean: Switch MAX_DETECTION_AGE ON / OFF. Default is ON.
    'MAX_ALERT_DETECTION_AGE': 604800, # Integer: Adjust the max_age alert time, to prevent TheHive triggers on alerts older than x amount of seconds. Default is 7 days.
    'HIVE_SERVER_IP': '192.168.1.15',  # String: The server IP or functioning DNS name of the TheHive instance you would like to query.
    'HIVE_SERVER_PORT': 9000,  # Integer: The TheHive port that's exposed to the instance this program will be running from.
    'HIVE_FQDN': 'http://192.168.1.15',  # String: The FQDN of the TheHive instance.
    'HIVE_API_KEY': 'iIMm25V63IjkoLN0MlsJJTcdrPYYhyBi',  # String: The TheHive API key generated for the API user you created.
    'LOG_FILE_LOCATION': 'debug.log'  # String: The file location where you would like to store your logs. You can specify a file path as well.
}

FLASK_SETTINGS = {
    'ENABLE_WEBSERVER': True,  # Boolean: Toggle to enable Flask to enrich the Slack integration. Set to False to disable.
    'FLASK_WEBSERVER_IP': '192.168.1.200',  # String: The IP of the instance you running this program on (will act as a webserver so needs to be reachable).
    'FLASK_WEBSERVER_PORT': 3000  # Integer: The webserver port you wish to expose Flask on.
}

TWILIO_SETTINGS = {
    'TWILIO_ENABLED': True,  # Boolean: Toggle to enable Twilio functionality. Set to False to disable.
    'TWILIO_SENDER': '+123456789',  # String: The Twilio number you wish to send from. Obtain from your Twilio console.
    'TWILIO_RTCP': ['+123456789', '+123456789'],  # List: A list of numbers you wish to send notifications via Twilio to.
    'ACCOUNT_SID': '',  # String: Your Twilio account SID. Obtain from Twilio console.
    'AUTH_TOKEN': '',  # String: Your Twilio auth token. Obtain from Twilio console.
    'TWIMLET_URL': 'http://twimlets.com/echo?Twiml=%3CResponse%3E%3CSay%3EHi+there.%3C%2FSay%3E%3C%2FResponse%3E'  # String: The FQDN to your custom Twimlet if you have one hosted. Create one here: https://www.twilio.com/labs/twimlets/echo.
}

SLACK_SETTINGS = {
    'SLACK_ENABLED': True,  # Boolean: Toggle to enable Slack functionality. Set to False to disable.
    'SLACK_APP_TOKEN': '',  # String: Your Slack application token. Obtain from your Slack console.
    'SLACK_CHANNEL': '',  # String: Your Slack application channel ID. Obtain from your Slack instance.
    'SLACK_WEBHOOK_URL': ''  String: Your Slack application webhook URL you generated. Obtain frmo your Slack instance.
}

Adjust what gets alerted on:

If you wish to adjust / add / remove what happens when a alert hits a specified SLA tier and the alert method that gets triggered (i.e, send via Twilio / send via Slack). You can adjust the definitions within configuration.py.`

How to install (Docker steps below, as well):

To do:

Create a slack application to receive messages to your workspace - follow this guide: https://github.com/slackapi/python-slack-sdk/blob/main/tutorial/01-creating-the-slack-app.md
Create a "echo" twimlet to playback a custom voice message when called by Twilio: https://www.twilio.com/labs/twimlets/echo
Create a twilio account to receive calls / sms's

  1. Install Python Virtual Environment python3 -m venv env
  2. Activate Virtual Environment . env/bin/activate
  3. Install dependencies pip install -r requirements.txt
  4. Add in custom variables in configuration.py which is injected into the main application at runtime.
  5. Run using python main.py - if you wish to run as a service, see below:

It's recommended to setup a reverse proxy to forward requests to your Python Flask server running (defined within configuration.py). Here's a mini example using Nginx. Note, this is not configured for production use. Replace x.x.x.x with the IP address you plan on having your application listen on.

server {
  listen 80;
  
  location /web_api/ {
    proxy_pass http://x.x.x.x:3000/;
  }
}

Alternatively you can use ngrok.com to tunnel if you do not wish to host a webserver on the instance the application runs on. However, this is discouraged due to the potential security risks associated with this.

Create sysinitv service on Linux (Debian):

If you would like to manage the application more easily, you can create a service to interact with the application.

Place in /etc/systemd/system/thehive_sla_monitor.service

[Unit]
Description=TheHive SLA Monitor

[Service]
Type=simple
ExecStart=/path/to/dir/env/bin/python /path/to/dir/main.py
WorkingDirectory=/path/to/dir/

[Install]
WantedBy=sysinit.target

Reload the daemon:

systemctl daemon-reload

Run / Start the bot with service thehive_sla_monitor start.

Deploy via Docker

If you wish to deploy via Docker, a handy Dockerfile has already been created for you. Just run the following commands in order:

Note: Adjust the --publish settings to the current IP address, exposed port (image built with port 3000) and container port if you build the image with a different port.

docker build -t thehive_sla_monitor .
docker run --publish 192.168.1.33:3000:3000 thehive_sla_monitor

thehive_sla_monitor's People

Contributors

dependabot[bot] avatar joshuasmeda avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Forkers

swipswaps

thehive_sla_monitor's Issues

General problems

Hello joshua,

I tried the commits, removed slack, twimlet etc, changed sms alert functions to add to 60min lists and dicts instead of 30min's

But there is a problem with it, it alerts old alerts on the hive, not recent alerts that are 30 mins old or 60 mins old.
another problem is that if I have it set to query severity based on 3, but lets say my defined critical word based alert is a medium alert. The script will not alert the defined word since it is a medium alert.

I really need to do the following:
1- Query severity high alerts and alert after when it hit hives 2 minutes later
2- Find for words in severity 2 alerts but only words not the rest
3- Alert in 2 minutes after the alert has been found only via sms

Enhancement | Sending sms based on what message contains

Hello Joshua,

Currently the script sends sms based on priority, but there are some cases where the alert can't remediate from high to critical, so maybe it will double check, prioritize severity of the alert and then looks at the artifact or the title to send sms even if it is medium or low. This is a very real use case scenario, do you think it will be possible?

King regards.

Feature | Sending SMS to multiple numbers

hello Joshua,

Is it possible to send multiple sms to multiple numbers, maybe defining numbers to send in an array and then putting the send message function in a loop until the array is done could solve this.

Could you help me with this request?

slackclient module is deprecated

Hello Joshua,

I think the slackclient module is deprecated, there is a module named "slack" but, it still wont work with the current script.

$ python3 bot.py 
Traceback (most recent call last):
  File "bot.py", line 13, in <module>
    from slackclient import SlackClient
ModuleNotFoundError: No module named 'slackclient'

Add max_timer

Implement a max_age timer of say, 6 days, and any alerts that are older than 6 days, we'll ignore? To be configurable through configuration.py and have a enabled/disabled feature.

Requested by @devatnull

Send sms only on one severity

Hello Joshua,

On my version of the code I only send severity 3 on twilio, I removed slack funcionality also,
We receive 7000 alerts daily, I can't notify on all of them, but only on custom severity, is it possible to remove slack notifications, and only sms on defined severity with your version?

What I use example:

    if response.status_code == 200:
        data = json.dumps(response.json())
        jdata = json.loads(data)
        for d in jdata:
            if d['severity'] == 3:
                ts = int(d['createdAt'])
                ts /= 1000
                alert_date = datetime.fromtimestamp(
                    ts).strftime('%Y-%m-%d %H:%M:%S')
                logging.info("Found: " + d['id'] + " " +
                             d['title'] + " " + str(alert_date))
                alert_date = datetime.strptime(alert_date, '%Y-%m-%d %H:%M:%S')
                diff = (current_date - alert_date)
                if sla_10 < diff.total_seconds() and sla_30 > diff.total_seconds():
                    logging.info("10/M SLA: " + str(diff) + " " + d['id'])
                    create_hive10_dict(d['id'], d['title'],
                                       str(alert_date), str(diff), d)
                    add_to_10m(d['id'])
for alert_id, alert_name in hive_10_dict.items():
        if alert_id in hive_10_list:
            logging.warning("Already sent SMS regarding ID: " + alert_id)
        else:
            if not is_empty(args):
                for alert_data in args:
                    alert_dump = json.dumps(alert_data, indent=2)
                    alert_json = json.loads(alert_dump)
                    #artifacts_list = []
                    artifactCount = 0
                    for artifact in alert_json['artifacts']:
                        artifactCount += 1
                    #  artifacts_list.append(artifact)
                    #artifacts = ' '.join([x for x in artifacts_list])
                    tags_list = []
                    for tag in alert_json['tags']:
                        tags_list.append(tag)
                    tags = ' '.join([x for x in tags_list])
                    sourceRef = (alert_json['sourceRef'])
                    source = (alert_json['source'])
                    title = (alert_json['title'])
                    description = (alert_json['description'])
                    #data_two_artifacts = (alert_json['artifacts'][1]['data'])
                    #dataType_two_artifacts = (alert_json['artifacts'][1]['dataType'])
                    #data_artifacts = (alert_json['artifacts'][0]['data'])
                    dataTypeList = []
                    #dataType_artifacts = (alert_json['artifacts'][0]['dataType'])
                    dataList = []
                    if artifactCount >= 4:
                        for i in range(4):
                            dataType_artifacts = (
                                alert_json['artifacts'][i]['dataType'])
                            data_artifacts = (
                                alert_json['artifacts'][i]['data'])
                            dataList.append(data_artifacts)
                            dataTypeList.append(dataType_artifacts)
                    else:
                        for i in range(artifactCount):
                            dataType_artifacts = (
                                alert_json['artifacts'][i]['dataType'])
                            data_artifacts = (
                                alert_json['artifacts'][i]['data'])
                            dataList.append(data_artifacts)
                            dataTypeList.append(dataType_artifacts)
                    dataTypes = '     '.join([x for x in dataTypeList])
                    datas = '     '.join([x for x in dataList])
            twilio_msg_body = 'The Hive' + "\n" + "SourceRef: " + \
                sourceRef + "\n" + "\n\n" + dataTypes + "\n\n" + datas
            for number in twilio_recipient:
                message = twilio_client.messages.create(
                    body=twilio_msg_body, from_=twilio_number, to=number)
                t.sleep(2)
                logging.info('SMS Sent: ' + str(message.sid))
                try:
                    add_to_10m(alert_id)
                except Exception as re:
                    logging.error(
                        "Error when adding to list after sending SMS: " + str(re))
            continue

Create spec tests

As the project grows, it's becoming more important to create tests (including mock tests) to ensure functionality doesn't break.

Enhancement | Sending messages in half

Hello joshua,

Another thing I noticed is that when a message is longer than 160 characters, twilio will give 30008 error on the panel, but the response to payload sent message will show empty or succeded but the SMS will not reach its destination.

So to help with this problem I added something like this:

if artifactCount >= 4:
                        for i in range(4):
                            dataType_artifacts = (
                                alert_json['artifacts'][i]['dataType'])
                            data_artifacts = (
                                alert_json['artifacts'][i]['data'])
                            dataList.append(data_artifacts)
                            dataTypeList.append(dataType_artifacts)
                    else:
                        for i in range(artifactCount):
                            dataType_artifacts = (
                                alert_json['artifacts'][i]['dataType'])
                            data_artifacts = (
                                alert_json['artifacts'][i]['data'])
                            dataList.append(data_artifacts)
                            dataTypeList.append(dataType_artifacts)

To explain what is above:

I put my longer messages that I need on hive but not on SMS I place them after 4th obversable so after 4th observable SMS will not be sent. But, I have some other cases that I have 3 Artifacts and some of them are very long, therefore the SMS will be failed to sent. I know this might be out of the use case but it is still a realistic situation that we are facing.

I thought of something like this:

If a payload is longer than 160 characters the bot will parse it, if it is longer than 160 characters, for example 320 characters it will cut it in half and send it after the first one.

What do you think about that? Could it be done?

Giving the choice to the user

Is it possible to correlate the alert timer, for example 2 minutes, also alerting notificitaion for example twilio sms based on severity?

If severity = 3
set SLA_SETTING = 120
set Alert_Notification = Twiliosms

If severity = 2
set SLA_SETTING = 600
set Alert_Notification = Slack

A correlational setting based on alert severity would leave many things to the user's hand. And, there is already an option to define severity 'SEVERITY_LEVEL': 2-4, SEVERITY_LEVEL':3-4, SEVERITY_LEVEL':1-3, etc. So instead of manually letting the developer choose which severity is which notification, this happens when the alert is this old etc, let the user define what they want, and I think this would make the code much simplier to understand. I believe turning it to something like this means removing 3-4 defined functions in the code correct? Along with on/off switch for alert-notifications since the alerting notification will be set on the correlational setting. Also, this along with the keyword based alert, cutting sms messages if it is too long and send the other half in another sms, this would be the perfect alerting script for TheHive project.

Please tell me your opinion on this.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.