mindphluxnet / cowrie-logviewer Goto Github PK

View Code? Open in Web Editor NEW

25.0 2.0 10.0 114 KB

A simple log viewer for cowrie

License: MIT License

Python 36.23% HTML 62.70% CSS 1.07%

cowrie honeypot log-viewer python webapp

cowrie-logviewer's Introduction

cowrie-logviewer

A simple log viewer for the cowrie honeypot.

Since the JSON logs are split by day, you can select the log to view in the top right corner of the page. Some rudimentary statistics are available as well. Uploaded payloads can also be viewed and downloaded via the "Uploaded files" page.

Attacker IPs are run against a local MaxMind GeoIP database to find out which country they belong to. The results are cached in a sqlite database.

Installation

Assuming you installed cowrie into your home directory:

su cowrie
cd ~/../cowrie
git clone https://github.com/mindphluxnet/cowrie-logviewer
cd cowrie-logviewer

If you want, you can make the script executable:

chmod +x cowrie-logviewer.py

Prerequisites

pip install -r requirements.txt

MaxMind GeoLite 2 Country database setup

mkdir maxmind
cd maxmind
wget http://geolite.maxmind.com/download/geoip/database/GeoLite2-Country.mmdb.gz
gunzip GeoLite2-Country.mmdb.gz
rm GeoLite2-Country.mmdb.gz

Configuration

Edit cowrie-logviewer.py to configure the script. There are several variables at the top of the file you can change:

"log_path" - the path to the cowrie log directory
"dl_path" - the path to the cowrie dl (downloads) directory
"maxmind_path" - the path to the MaxMind GeoLite 2 Country database. Default is "maxmind/GeoLite2-Country.mmdb"
"bind_host" - the IP address the web server should bind to, default 0.0.0.0
"bind_port" - the port the web server should listen to, default 5000
"min_upload_size" - min. file size in bytes to be to be listed on the "Uploaded files" page. Default is 1024
"debug" - if you want debug messages, set this to True. Default "False"
"use_gzip" - if you don't want gzip compression, set this to False. Default "True"
"filter_events" - a list of log events to filter. Default "[ 'cowrie.direct-tcpip.request', 'cowrie.direct-tcpip.data' ]"

Usage

python cowrie-logviewer.py

or (if you made the script executable)

./cowrie-logviewer.py

Once it's running, open

http://yourhostname:yourport

in the web browser of your choice. The default port is 5000.

Event filtering

Some log events can be quite annoying since they provide very little insight into what's actually happening but take up huge amounts of log lines. By default, cowrie-logviewer filters out events related to tcp/ip forwarding. I implemented this after having some idiot attempt to abuse my honeypot for some kind of tcp/ip forwarding exploit for about 24 hours straight. This caused the logs to become almost unreadable. Filtering these events removes that annoyance. Of course, the actual log files still contain these events so nothing is lost.

Local network connections

Since local network IPs aren't in the MaxMind database for obvious reasons, there's no flag icon attached to these connections. This isn't a bug, it's a feature.

cowrie-logviewer's People

Contributors

Stargazers

Watchers

Forkers

ltw-gcr-csoc steveclement msymt paulslewis66 rasulovk breenet vineetdwivedi11 realradioactive edvinbaggman

cowrie-logviewer's Issues

Move configuration options to a separate config file

Instead of having the user edit the .py file, there should be an .ini file for configuration options.

Add more statistics

There should be more statistics, such as:

most used usernames
most used passwords
(maybe) most used user/pass combinations
attacks over time graph (this would mean additional database work since logs are split by day)

I'd also really like to have the viewer analyze sessions to point out specific attack patterns (for example Mirai) but this would require more research.

Login with username and password?

Is it possible? I would like to have a login in the log viewer.

Suggestion: Most used IP adresses with Heatmap

Is it possible to add a column on the Attackers by Country page with the IP addresses between the country names and the IP Count? Included a heatmap where exactly these IP's are located?
So you have instead of the graph on the right side an embedded Heatmap, e.g. an offline version of openheatmap.com build on your one or based on MaxMind GeoIP, with the located IP's?!

How would you think about it? (easy) possible or not? :)

I cannot view my data on the webpage

Hello,

I'm a novice with cowrie and the visualization tool, so mind my novice question :)
I've already setup the tool following your guide, but got stuck when it comes to this step:

Once it's running, open

http://yourhostname:yourport

in the web browser of your choice. The default port is 5000.

The cowrie-logviewer.py is already running, but when I tried opening my browser using my hostname and the default port, I got a "This site can’t be reached, took too long to respond.".

Can anyone kindly advice on what did I do wrong? Thank you.

Daemonize the script

The script should really be able to run in the background as a daemon if the user wants it to.

Geolitecity or country in settings?

Hello,

Playing around with your viewer and noticed an issue. The instructions say to download:

wget http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz

But in the cowrie-logviewer.py, the maxmind path points to GeoLite2-Country.mmdb. I'm assuming City is the correct file?

No usernames and passwords shown in logviewer

Hi I recently installed cowrie and your logviewer on my maspberry pi.
Fenerally everything works fine i can see the IP-addresses of the attackers and their country of origin. However the "Userrname and Password" page under "Statistics" is blank though the logfile lists successfull attempts.

Can you give me a clue where to look in order to fix it?

Optimize log page render time

Currently, every IP address in a log is checked against the database every time a log page is rendered. If its not in the database, it'll be fetched from ipapi.co. This needs to be optimized since right now it's taking ages (about a minute on my Atom powered machine).

Ideas to test:

put all IPs in a dictionary and use this as lookup table. If an IP is already in there it will no longer be checked. I've tested an implementation of this idea but it didn't improve loading times, probably because it's still hundreds, if not thousands of IPs that need to be checked per log file.
constantly parse logs in the background and add IPs to the database so loading isn't stalled by the ipapi.co lookup. Should improve things, but even a log with already known IPs loads rather slowly.

Country error

[2018-04-13 15:42:56,128] ERROR in app: Exception on /stats/countries [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "./cowrie-logviewer.py", line 147, in show_stats_countries
    tmp = [ pycountry.countries.get(alpha_2=country[0]).name, country[0], country[1] ]
  File "/usr/local/lib/python2.7/dist-packages/pycountry/db.py", line 43, in load_if_needed
    return f(self, *args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/pycountry/db.py", line 101, in get
    return self.indices[field][value]
KeyError: None

KeyError: 'alpha_2'

Ok everything was setup and was working with the log viewer and now all of a sudden
today I am getting a key error.

The only time I get this error is when I go to Statistics and then attacks by country.

Like I said it was working fine now today I am getting this..

Running Debian 9

uname -a
Linux server1 4.9.0-6-amd64 #1 SMP Debian 4.9.88-1+deb9u1 (2018-05-07) x86_64 GNU/Linux

python -V
Python 2.7.13

pip install -r requirements.txt
Requirement already satisfied: flask in /usr/lib/python2.7/dist-packages (from -r requirements.txt (line 1))
Requirement already satisfied: ipapi in /usr/local/lib/python2.7/dist-packages/ipapi-0.5.2-py2.7.egg (from -r requirements.txt (line 2))
Requirement already satisfied: pycountry in /usr/lib/python2.7/dist-packages (from -r requirements.txt (line 3))
Requirement already satisfied: path.py in /usr/lib/python2.7/dist-packages (from -r requirements.txt (line 4))
Requirement already satisfied: flask-compress in /usr/lib/python2.7/dist-packages (from -r requirements.txt (line 5))
Requirement already satisfied: python-dateutil in /usr/lib/python2.7/dist-packages (from -r requirements.txt (line 6))
Requirement already satisfied: geoip2 in /usr/lib/python2.7/dist-packages (from -r requirements.txt (line 7))
Requirement already satisfied: requests[security] in /usr/lib/python2.7/dist-packages (from ipapi->-r requirements.txt (line 2))
Requirement already satisfied: ipaddress in /usr/lib/python2.7/dist-packages (from geoip2->-r requirements.txt (line 7))
Requirement already satisfied: pyOpenSSL>=0.14 in /usr/lib/python2.7/dist-packages (from requests[security]->ipapi->-r requirements.txt (line 2))
Requirement already satisfied: cryptography>=1.3.4 in /usr/lib/python2.7/dist-packages (from requests[security]->ipapi->-r requirements.txt (line 2))
Requirement already satisfied: idna>=2.0.0 in /usr/lib/python2.7/dist-packages (from requests[security]->ipapi->-r requirements.txt (line 2))

This was the dump that I got

KeyError

KeyError: 'alpha_2'
Traceback (most recent call last)

File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1997, in __call__

return self.wsgi_app(environ, start_response)

File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1985, in wsgi_app

response = self.handle_exception(e)

File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1540, in handle_exception

reraise(exc_type, exc_value, tb)

File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1982, in wsgi_app

response = self.full_dispatch_request()

File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1614, in full_dispatch_request

rv = self.handle_user_exception(e)

File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1517, in handle_user_exception

reraise(exc_type, exc_value, tb)

File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1612, in full_dispatch_request

rv = self.dispatch_request()

File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1598, in dispatch_request

return self.view_functions[rule.endpoint](**req.view_args)

File "/home/cowrie/cowrie-logviewer/cowrie-logviewer.py", line 147, in show_stats_countries

tmp = [ pycountry.countries.get(alpha_2=country[0]).name, country[0], country[1] ]

File "/usr/lib/python2.7/dist-packages/pycountry/db.py", line 123, in get

return self.indices[field][value]

KeyError: 'alpha_2'

The debugger caught an exception in your WSGI application. You can now look at the traceback which led to the error.

To switch between the interactive traceback and the plaintext one, you can click on the "Traceback" headline. From the text traceback you can also create a paste of it. For code execution mouse-over the frame you want to debug and click on the console icon on the right side.

You can execute arbitrary Python code in the stack frames and there are some extra helpers available for introspection:

dump() shows all variables in the frame
dump(obj) dumps all that's known about the object