1ultimat3 / pcapviz Goto Github PK

Visualize network topologies and collect graph statistics based on pcap files

Python 100.00%

network-topology python pcap security-tools

pcapviz's Introduction

PcapViz

PcapViz draws networks as device topologies and as information flows using the packet information in pcap files captured from a network device using tcpcap or other capture software. It filters and optionally displays the captured packets at any one of 3 "layers". These are:

device level traffic topology,
ip communication and
tcp/udp communication

Each yields a distinct network graph from the same set of network packets. This separation makies it much easier to see the data flows at each level rather than mixing them up as many other visualisation packages do. It should be possible to determine key topological nodes or to spot patterns of data exfiltration attempts more easily.

Features

Network topology graphs - 2 = device; conversation information flow graphs: 3 = ip, 4 = tcp/udp
Communication graph node labels show country information and connection stats
Lists the most frequently contacted and frequently sending machines
Node labels include the host domain name if available from a reverse DNS lookup.
command line choice of Graphviz graph layout engine such as dot or sfdp.

Usage

usage: main.py [-h] [-i [PCAPS [PCAPS ...]]] [-o OUT] [-g GRAPHVIZ] [--layer2]
               [--layer3] [--layer4] [-fi] [-fo] [-G GEOPATH] [-l GEOLANG]
               [-E LAYOUTENGINE] [-s SHAPE]

pcap topology and message mapper

optional arguments:
  -h, --help            show this help message and exit
  -i [PCAPS [PCAPS ...]], --pcaps [PCAPS [PCAPS ...]]
                        space delimited list of capture files to be analyzed
  -o OUT, --out OUT     topology will be stored in the specified file
  -g GRAPHVIZ, --graphviz GRAPHVIZ
                        graph will be exported to the specified file (dot
                        format)
  --layer2              device topology network graph
  --layer3              ip message graph. Default
  --layer4              tcp/udp message graph
  -fi, --frequent-in    print frequently contacted nodes to stdout
  -fo, --frequent-out   print frequent source nodes to stdout
  -G GEOPATH, --geopath GEOPATH
                        path to maxmind geodb data
  -l GEOLANG, --geolang GEOLANG
                        Language to use for geoIP names
  -E LAYOUTENGINE, --layoutengine LAYOUTENGINE
                        Graph layout method - dot, sfdp etc.
  -s SHAPE, --shape SHAPE
                        Graphviz node shape - circle, diamond, box etc.

Examples from running tests/core.py on the test.pcap file

Drawing a communication graph (layer 2), segment

python main.py -i tests/test.pcap -o test2.png --layer2

Layer3 with default sfdp layout

Layer4 with default sfdp layout

Return hosts with largest numbers of incoming packets:

python3 main.py -i tests/test.pcap -fi --layer3
4 172.16.11.12
1 74.125.19.17
1 216.34.181.45 slashdot.org
1 172.16.11.1
1 96.17.211.172 a96-17-211-172.deploy.static.akamaitechnologies.com

Installation

Required:

GraphViz See system notes below
Pip package requirements The Maxmind Python API and other dependencies will be installed when you run:
```
pip3 install -r requirements.txt
```
so of course, please run that! You are using a python virtual environment aren't you?

Installation Debian

For Debian-based distros you have to install GraphViz with some additional dependencies:

apt-get install python3-dev
apt-get install graphviz libgraphviz-dev pkg-config

Installation OSX

Scapy does not work out-of-the-box on OSX. Follow the platform specific instruction from the scapy website

brew install graphviz
brew install --with-python libdnet
brew install https://raw.githubusercontent.com/secdev/scapy/master/.travis/pylibpcap.rb

Testing

Unit tests can be run from the tests directory:

python3 core.py

The sample images above are the test output graphs.

Note that there are at present 2 warnings about deprecated features in graphviz and for tests to work, you may need to adjust the fake args to point to your copy of the geoIP data file. Without access to the geoIP data, two of the tests will always fail.

pcapviz's People

Contributors

Stargazers

Watchers

Forkers

zerowindow furetosan wflk barryirwin mgcfish congto openstacks jinyu00 songofhack johnjohnsp1 securitywarrior rootjacker m41doror awesome-security trietptm-on-coding-algorithms team-firebugs gbiagomba expertasif grukz monkeysm8 fo0nikens zard777 ro9ueadmin cybertaoflow hack404-007 duggore tim1512 hathill bhaprayan nluedtke 4n6strider indigos33k3r jinnu92 daydayup40 lucasdiniz askylyz ting2-wang sts0mrg0 consokodev blackhook fubar2 evertonteotonio 5l1v3r1 opsec-scalefocus zobber olderlong th3architect davidestrada99 pyonghe jorson-chen senevren ai-abdellah zakariae48 hartl3y94 cqr-cryeye-forks zain93393 adonayteshome acbogu

pcapviz's Issues

FQDN & DNS Resolution

Hello,

This is not an "issue" more of a request and question. But I was wondering if the tool supported DNS resolution for IPs or if the tool can provide the FQDN of the IP address? If not, what modifications would I need to do to make it do so? Or would you mind making the modification to enable this feature? Either way it would be greatly appreciated!

GeoIP.dat is not available anymore

Provider has moved to GeoLite2. Would it be possible to update? I'd like to check out your tool ;)

Error in core.py on line 42

Hi,

I'm trying to use this python script (on Windows).

Commande line :

python .\main.py -i "input.pcap" -o "output.png" --layer2

Error:

Traceback (most recent call last):
File ".\main.py", line 35, in
g = GraphManager(packets, layer=layer)
File "P:\Python\PcapViz\pcapviz\core.py", line 42, in init
self.graph.add_edge(src, dst, {'packets': [packet]})
TypeError: add_edge() takes 3 positional arguments but 4 were given

Requirements not installing correctly

When I try to install the requirements using the requirements.txt file I get the following errors:

Failed building wheel for pygraphpviz

issues

Traceback (most recent call last):
File "main.py", line 3, in
from pcapviz.core import GraphManager
File "/home/lau/PcapViz/pcapviz/core.py", line 8, in
from pygeoip import GeoIP
ImportError: No module named pygeoip

is there any chance that someone can take my pcap file and produce a layer2 and layer3 topology png in the meantime. I really need it at the moment.

TypeError: 'errors' is an invalid keyword argument for this function

I install PcapViz on my ubuntu 14.04
When I run the tset script python main.py -i smallFlows.pcap -o small_tcp.png --layer3
it returns
Traceback (most recent call last):
File "main.py", line 3, in <module>
from pcapviz.core import GraphManager
File "/home/lin/PcapViz/pcapviz/__init__.py", line 1, in <module>
from scapy.all import *
File "/usr/local/lib/python2.7/dist-packages/scapy/all.py", line 10, in <module>
from .config import *
File "/usr/local/lib/python2.7/dist-packages/scapy/config.py", line 11, in <module>
from .data import *
File "/usr/local/lib/python2.7/dist-packages/scapy/data.py", line 184, in <module>
TCP_SERVICES,UDP_SERVICES=load_services("/etc/services")
File "/usr/local/lib/python2.7/dist-packages/scapy/data.py", line 109, in load_services
f=open(filename, errors='ignore')
TypeError: 'errors' is an invalid keyword argument for this function
How can i solve this? Thanks

graph is too large

Hello,

great tool you have written.
Can you tell me how to solve the following issue?
RuntimeWarning: neato: graph is too large for cairo-renderer bitmaps. Scaling by 0.198021 to fit
warnings.warn(b"".join(errors).decode(self.encoding), RuntimeWarning)

Thanks in advance

John

Layer 4 fails when output is ~

FYI:

Traceback (most recent call last):
  File "main.py", line 67, in <module>
    sg.draw(filename = ofn)
  File "/media/cuckoo/opt/PcapViz/pcapviz/core.py", line 198, in draw
    graph.draw(filename)
  File "/usr/local/lib/python2.7/dist-packages/pygraphviz/agraph.py", line 1518, in draw
    fh = self._get_fh(path, 'w+b')
  File "/usr/local/lib/python2.7/dist-packages/pygraphviz/agraph.py", line 1547, in _get_fh
    fh = open(path, mode=mode)
IOError: [Errno 2] No such file or directory: 'Raw_223_/home/me/bad/work/pcaps/layer4.png'

layer 2 and 3 had no issues outputing to ~/bad/work/pcaps/layer3.png

the lib 'networkx' has been updated

The lib 'networkx' has been updated to ver 2.0 in Sep, 2017(https://pypi.python.org/pypi/networkx/), which the version to download use 'pip install'. And the parameters of some functions have changed. For example the parameters of add_edges() in DiGraph have changed to 'def add_edge(self, u, v, **attr)' from 'def add_edge(self, u, v, attr_dict=None, **attr)'. So when I compile the source code, some errors occur like 'TypeError: add_edge() takes exactly 3 arguments (4 given)'.

Regarding Visualization in Gephi

How can I visualize the graph (dot file) obtained from .pcap files in Gephi ?

Thoughts on this attr issue I’m running into?

Installing the requirement of graphviz

When I am trying to install it, using the graphviz I can this issue:

Error! can't create out file

great tool.thank you so much for develop this.
i got error

$ python main.py -i smallFlows.pcap -o small_tcp_l2.png --layer3
WARNING: No route found for IPv6 destination :: (no default route?)
WARNING:root:could not load GeoIP data
Traceback (most recent call last):
File "main.py", line 38, in
g.draw(filename=args.out)
File "/home/twster/PcapViz/pcapviz/core.py", line 116, in draw
graph = self.get_graphviz_format()
File "/home/twster/PcapViz/pcapviz/core.py", line 144, in get_graphviz_format
agraph = networkx.to_agraph(self.graph)
AttributeError: 'module' object has no attribute 'to_agraph'
twster@hckndev:~/PcapViz$