omriher / captipper Goto Github PK

--- CapTipper-master/pcapparser/constant.py 2015-08-06 16:42:26.000000000 -0700
+++ CapTipper-fix/pcapparser/constant.py 2015-08-13 15:28:32.170656700 -0700
@@ -14,6 +14,7 @@ class LinkLayerType(object):
"""LinkType"""
ETHERNET = 1
LINUX_SLL = 113

• RAWIP = 101

--- CapTipper-master/pcapparser/packet_parser.py 2015-08-06 16:42:26.000000000 -0700
+++ CapTipper-fix/pcapparser/packet_parser.py 2015-08-13 15:31:03.883334100 -0700
@@ -78,6 +78,10 @@ def dl_parse_ethernet(link_packet):
pass
return n_protocol, link_packet[eth_header_len:]

+def dl_parse_rawip(link_packet):

• """parse raw ip packet"""
• raw_ip_header_len=0
• return NetworkProtocol.IP,link_packet

def dl_parse_linux_sll(link_packet):
@@ -178,6 +182,8 @@ def get_link_layer_parser(link_type):
return dl_parse_ethernet
elif link_type == LinkLayerType.LINUX_SLL:
return dl_parse_linux_sll

• elif link_type == LinkLayerType.RAWIP:

•    return dl_parse_rawip
  

  else:
  return None

I just installed this and am supposed to run open 5 at CT> for this horribly put together class. It wont work because running firefox as root in a regular user session etc , anyone know a fix for this

I'm getting this error,

CapTipper v0.3 b14 - Malicious HTTP traffic explorer tool
Copyright 2015 Omri Herscovici <[email protected]>

[A] Analyzing PCAP: ../../Downloads/pcap/2020-12-31-traffic-analysis-quiz-01.pcap
int() argument must be a string, a bytes-like object or a number, not '_collections._tuplegetter'

The pcap can be downloaded here https://www.malware-traffic-analysis.net/2020/12/31/2020-12-31-traffic-analysis-quiz-6-pcaps.zip, specifically 2020-12-31-traffic-analysis-quiz-01.pcap. Any idea how to solve this? I'm running python 3.9.2.

Hi, thanks for this great tool!

I'm having some issues reading pcap files generated by Virtualbox.

For example the pcap file:
https://mcfp.felk.cvut.cz/publicDatasets/CTU-Malware-Capture-Botnet-14/2013-10-18_capture-win15.pcap

./CapTipper.py 2013-10-18_capture-win15.pcap 1234
CapTipper v0.1 - Malicious HTTP traffic explorer tool
Copyright 2015 Omri Herscovici [email protected]

[A] Analyzing PCAP: 2013-10-18_capture-win15.pcap
unpack requires a string argument of length 14

The issue is that there is a first (maybe broken) packet in the pcap file that is stopping the parser. To temporary solve this issue I must delete the first packet in the pcap file, with the following command:

editcap 2013-10-18_capture-win15.pcap 2013-10-18_capture-win15.NEW.pcap 1

Usually tcpdump/wireshark ignore this type of situation so you can read the pcap files. Not sure why.
Maybe there is a way of telling CapTipper to ignore it also?

thanks
sebas

In tcp stream 2, 3, and 4 there are binaries that have content type <application/x-msdownload>.
Captipper finds them pretty fine, however neither <dump all> nor <-d> switch does not export those files.

In addition to that there is also another bug in this sample.
There are two requests to the following URL path, however CapTipper catches only one of them, particularly the first one.

URL

/?es_sm=108&oq=xfR7L7VUbwq0hBfTewFllYxYA1pGoauojkXQnEOd1JGK_xWJYAsR96KlJLR_mhj2&aqs=chrome.113j102.406q9m8&q=w3rQMvXcJxvQFYbGMvnDSKNbNk_WHViPxo6G9MildZ-qZGX_k7PDfF-qoVvcCgWR&sourceid=chrome&ie=Windows-1252 

Sample

http://www.malware-traffic-analysis.net/2016/12/13/2016-12-13-pseudoDarkleech-Rig-V-sends-Cerber-ransomware.pcap.zip

While using CapTipper for urls longer than 1024 characters. By changing length in CapTipper-master\CTServer.py at line 120 to 4096 seems to be working for me. Can anyone check this?

original line:
self.data = self.request.recv(1024).strip()

modified line:
self.data = self.request.recv(4096).strip()

Having an issue after a recent upgrade.

Captipper works fine as along as capinfos shows the filetype as : 'Wireshark/tcpdump... - libpcap' but not if it's 'Wireshark - nanosecond libpcap'

See details below.

Works

File name:           tmp1.pcap
File type:           Wireshark/tcpdump/... - pcap
File encapsulation:  Ethernet
Packet size limit:   file hdr: 65535 bytes
Number of packets:   374 
File size:           389 kB
Data size:           383 kB
Capture duration:    36 seconds
Start time:          Thu Mar  3 09:37:30 2016
End time:            Thu Mar  3 09:38:06 2016
Data byte rate:      10 kBps
Data bit rate:       85 kbps
Average packet size: 1024.82 bytes
Average packet rate: 10 packets/sec
SHA1:                3f5cdb3731a1c995959c3a4edd66168f03d96096
RIPEMD160:           e8b732f88061521a9c7b2de5d428de4b05bf945e
MD5:                 1168b1ff64f5c4d540a9e371c0d7ebff
Strict time order:   True

Does not work

File name:           tmp.pcap
File type:           Wireshark - nanosecond libpcap
File encapsulation:  Ethernet
Packet size limit:   file hdr: 1536 bytes
Number of packets:   8 
File size:           1264 bytes
Data size:           1112 bytes
Capture duration:    22 seconds
Start time:          Thu Mar  3 09:26:32 2016
End time:            Thu Mar  3 09:26:54 2016
Data byte rate:      49 bytes/s
Data bit rate:       396 bits/s
Average packet size: 139.00 bytes
Average packet rate: 0 packets/sec
SHA1:                5c41dfee0f69d5562d960fba8a064ad17e186aeb
RIPEMD160:           726ca7ba2c233b968ac3d0e19c380059a622679b
MD5:                 ec922f94e3d98e6bca066d75c65ce24e
Strict time order:   True

Actual Error message:

~/Desktop/CapTipper $ python CapTipper.py tmp.pcap
CapTipper v0.3 b11 - Malicious HTTP traffic explorer tool
Copyright 2015 Omri Herscovici [email protected]

[A] Analyzing PCAP: tmp.pcap
unknown file format.

python3 /opt/Malware-Project/tools/CapTipper/CapTipper.py 2017-2-20_win10.pcap -r .
CapTipper v0.3 b14 - Malicious HTTP traffic explorer tool
Copyright 2015 Omri Herscovici [email protected]

[A] Analyzing PCAP: 2017-2-20_win10.pcap
unpack requires a buffer of 14 bytes
ERROR:root:Traceback (most recent call last):
File "/opt/Malware-Project/tools/CapTipper/pcapparser/packet_parser.py", line 194, in read_tcp_packet
state, pack = read_tcp_pac(link_packet, link_layer_parser)
File "/opt/Malware-Project/tools/CapTipper/pcapparser/packet_parser.py", line 135, in read_tcp_pac
state, source, dest, tcp_packet, src_mac = read_ip_pac(link_packet, link_layer_parser)
File "/opt/Malware-Project/tools/CapTipper/pcapparser/packet_parser.py", line 102, in read_ip_pac
n_protocol, ip_packet = link_layer_parser(link_packet)
File "/opt/Malware-Project/tools/CapTipper/pcapparser/packet_parser.py", line 67, in dl_parse_ethernet
(n_protocol, ) = struct.unpack(b'!12xH', ethernet_header)
struct.error: unpack requires a buffer of 14 bytes

^Cint() argument must be a string, a bytes-like object or a number, not '_collections._tuplegetter'

You can try with this pcap https://mcfp.felk.cvut.cz/publicDatasets/CTU-Malware-Capture-Botnet-371-1/2017-2-20_win10.pcap

CapTipper v0.2 b09 - Malicious HTTP traffic explorer tool
Copyright 2015 Omri Herscovici [email protected]

[A] Analyzing PCAP: CTU-Malware-Capture-Botnet-64//2014-04-07_capture-win6.pcap

[+] Traffic Activity Time: Sun, 02/16/75 12:12:09
[+] Conversations Found:

0: / -> text/html (0.html) [0.0 B]
1: /(2) -> text/html ((2)) [122.9 KB](Magic: HTML)

[!] Generating Reports...
[+] Created JSON report to CTU-Malware-Capture-Botnet-64/2014-04-07_capture-win6.json
[E] Error creating HTML report in CTU-Malware-Capture-Botnet-64/2014-04-07_capture-win6.html : 'USER-AGENT' is not defined

Near: [(<function _DoSection at 0x7fe1d6af3410>,

Have you considered a method of adding plugins/modules?

I have some scripts i use to de-obfuscate rig EK and a couple of others. Would love to add them along with some basic html analysis tools

First I love this project and looking forward to its development.
I found an issue running the hosts command on a Linux machine results in the following;

CT> hosts
Found Hosts:

 www.bing.com

Exiting CapTipper
WebServer Shutdown.

The issue is with the unichr options.

on windows they are fine

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Users\Admin>python
Python 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)] on win
32
Type "help", "copyright", "credits" or "license" for more information.
>>> print unichr(9500)
├
>>> print unichr(9492)
└
>>>

On Linux not so much

root@viper:~/github/CapTipper# python
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> print unichr(9500)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character u'\u251c' in position 0: ordinal not in range(128)
>>> print unichr(9492)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2514' in position 0: ordinal not in range(128)
>>>

A Simple fix is to replace the chars with a |

 for host_uri in hosts[host]:
            print " " + "|"  + "-- " + host_uri.encode('utf8')

Which then outputs the expected results

CT> hosts
Found Hosts:

 www.bing.com
 |-- /fd/ls/GLinkPing.aspx?IG=aee5908ea2d64991aa8b8996fd170a75&&ID=SERP,5091.1   [0]
 |-- /fd/ls/lsp.aspx   [36]


 www.ciniholland.nl
 |-- /wp-content/plugins/contact-form-7/includes/css/styles.css?ver=3.7.2   [1]
 |-- /wp-content/themes/cini/js/functions.js   [2]
 |-- /wp-content/plugins/sitemap/css/page-list.css?ver=4.2   [3]
 |-- /   [4]

Or use a try / except to display | if unichr fails

Ill send a pull request when I get home

Testing the python3_support branch of CapTipper and attempted to follow your walkthrough to the letter. Got to the dump phase and tried both the dump all /tmp/ -e method and the CapTipper.py 2014-11-06-Nuclear-EK-traffic.pcap -d /tmp/ option and both got the same errors:

root@fb16f3336d75:/captipper# ./CapTipper.py 2014-11-06-Nuclear-EK-traffic.pcap -d /tmp 
CapTipper v0.3 b14 - Malicious HTTP traffic explorer tool
Copyright 2015 Omri Herscovici <[email protected]>

[A] Analyzing PCAP: 2014-11-06-Nuclear-EK-traffic.pcap

[+] Traffic Activity Time:  Thu, 11/06/14 15:02:35
[+] Conversations Found:

0:  /  -> text/html (0.html) [5.4 KB]  (Magic: GZ)
1:  /wp-includes/js/jquery/jquery.js?ver=1.7.2  -> application/javascript (jquery.js) [38.6 KB]  (Magic: GZ)
2:  /seedadmin17.html  -> text/html (seedadmin17.html) [354.0 B]  (Magic: HTML)
3:  /wp-content/uploads/2014/01/MetroWest_COVER_Issue2_Feb2014.jpg  -> image/jpeg (MetroWest_COVER_Issue2_Feb2014.jpg) [341.8 KB]  (Magic: JPG)
4:  /15c0b14drr9f_1_08282d03fb0251bbd75ff6dc6e317bd9.html  -> text/html (15c0b14drr9f_1_08282d03fb0251bbd75ff6dc6e317bd9.html) [110.5 KB]  (Magic: HTML)
5:  /images/footer/3000melbourne.png  -> image/png (3000melbourne.png) [2.9 KB]  (Magic: PNG)
6:  /images/footer/3207portmelbourne.png  -> image/png (3207portmelbourne.png) [3.0 KB]  (Magic: PNG)
7:  /wp-content/uploads/2012/09/background1.jpg  -> image/jpeg (background1.jpg) [32.3 KB]  (Magic: JPG)
8:  /00015d76d9b2rr9f/1415286120  -> application/octet-stream (00015d76.swf) [30.8 KB]  (Magic: SWF)
9:  /00015d766423rr9f/1415286120  -> application/pdf (XykpdWhZZ2.pdf) [9.7 KB]  (Magic: PDF)
10:  /00015d76rr9f/1415286120/5/x00809070554515d565b010b03510053535c0505;1;6  -> application/octet-stream (5.exe) [136.0 KB]  (Magic: EXE)
11:  /00015d76rr9f/1415286120/5/x00809070554515d565b010b03510053535c0505;1;6;1  -> application/octet-stream (5.exe) [136.0 KB]  (Magic: EXE)
12:  /00015d76rr9f/1415286120/7  -> application/octet-stream (7.exe) [136.0 KB]  (Magic: EXE)
13:  /00015d761709rr9f/1415286120  -> application/octet-stream (00015d76.swf) [7.9 KB]  (Magic: XAP)
14:  /00015d76rr9f/1415286120/8  -> application/octet-stream (8.exe) [136.0 KB]  (Magic: EXE)

 GZIP Decompression of object 0 (0.html) successful!
 New object created: 15

 GZIP Decompression of object 1 (jquery.js) successful!
 New object created: 16

[Errno 21] Is a directory: '/tmp/0-0.html'
[Errno 21] Is a directory: '/tmp/1-jquery.js'
[Errno 21] Is a directory: '/tmp/2-seedadmin17.html'
[Errno 21] Is a directory: '/tmp/3-MetroWest_COVER_Issue2_Feb2014.jpg'
[Errno 21] Is a directory: '/tmp/4-15c0b14drr9f_1_08282d03fb0251bbd75ff6dc6e317bd9.html'
[Errno 21] Is a directory: '/tmp/5-3000melbourne.png'
[Errno 21] Is a directory: '/tmp/6-3207portmelbourne.png'
[Errno 21] Is a directory: '/tmp/7-background1.jpg'
[Errno 21] Is a directory: '/tmp/8-00015d76.swf'
[Errno 21] Is a directory: '/tmp/9-XykpdWhZZ2.pdf'
[Errno 21] Is a directory: '/tmp/10-5.exe'
[Errno 21] Is a directory: '/tmp/11-5.exe'
[Errno 21] Is a directory: '/tmp/12-7.exe'
[Errno 21] Is a directory: '/tmp/13-00015d76.swf'
[Errno 21] Is a directory: '/tmp/14-8.exe'
[Errno 21] Is a directory: '/tmp/15-ungzip-0.html'
[Errno 21] Is a directory: '/tmp/16-ungzip-jquery.js'

While the copy/paste from this test is from a Docker (Ubuntu 20.04), I have tested this in a physical Ubuntu 18.04 installation, and both a virtual 18.04 VM and 20.04 VM. The only change made to both was adding the '3' at the end of the shebang in CapTipper.py.

I did change the 'cgi.escape' to 'html.escape' in the Ubuntu 20 install, but this is only part of the jsontemplate, and does not (should not) affect the dump_all_files or dump_file function in CTCore.py

Hi. I was analyzing the pcap file https://mcfp.felk.cvut.cz/publicDatasets/CTU-Malware-Capture-Botnet-35-1/2014-01-31_capture-win7.pcap from our dataset and for some reason Captipper is not finishing. I run it in a fast computer for more than 10 hours.
Other pcaps are finishing correctly, but not this one. And it is not so big.
Do you know what could be be issue?

thanks
sebas

In this capture there are two clients infected: 192.168.0.250 and 192.168.0.251, however only one is detected and used in the report.

http://mcfp.felk.cvut.cz/publicDatasets/CTU-Malware-Capture-Botnet-116-2/2012-05-25-captura-2.html
http://mcfp.felk.cvut.cz/publicDatasets/CTU-Malware-Capture-Botnet-116-2/2012-05-25-captura-2.pcap

unsupported operand type(s) for &: 'str' and 'int'

Attempting to execute script from different location and html report fails to write, but json report has no issue.

[root@deeznuts uploads]# pwd
/home/appdev/www/uploads

/usr/bin/python /home/appdev/CapTipper-master/CapTipper.py pcap.pcap -r /var/www/html/uploads/

[!] Generating Reports...
[+] Created JSON report to /var/www/html/uploads/pcap.json
[E] Error creating HTML report in /var/www/html/uploads/pcap.html : [Errno 2] No such file or directory: 'jsontemplate/CapTipperTemplate.html'
[E] Failed creating HTML report

If I execute script from master directory containing jasontemplate no errors.

This pcap file:
https://mcfp.felk.cvut.cz/publicDatasets/CTU-Malware-Capture-Botnet-112-2/2015-04-09_capture-win11.pcap

[-] Error parsing body of uri: http://exist.ru/price.aspx?&pcode=s5304110 : CRC check failed 0x28514949 != 0xafdd2ee9L

And it didn't finished in 2 days.

Exception RuntimeError: 'generator ignored GeneratorExit' in <generator object read_tcp_packet at 0xb6d5639c> ignored
[-] Error parsing body of uri: /plugins/like.php?api_key=160644017296185&locale=en_US&sdk=joey&channel_url=http%3A%2F%2Fstatic.ak.facebook.com%2Fconnect%2Fxd_arbiter.php%3Fversion%3D18%23cb%3Dfd2bd148c17a68%26origin%3Dhttp%253A%252F%252Fes.gossipcenter.com%252Ff17cd0a9175c26b%26domain%3Des.gossipcenter.com%26relation%3Dparent.parent&href=http%3A%2F%2Fwww.facebook.com%2Fgossipcenter&node_type=link&width=90&font=verdana&layout=button_count&colorscheme=light&action=like&show_faces=false&send=false&extended_social_context=false : unpack requires a string argument of length 4

Hi.
I was wondering if there is some plan to show the HTTP method (GET or POST or whatever) that was originally sent and the host name in the output of the convs command.

Now if I made convs, I got something like:
CT> convs
Conversations Found:
0: /cloud.html -> text/html (cloud.html) [1B]
1: /navigate.xml -> text/xml (navigate.xml) [251 B]
2: / -> (2.html) [0 B]
3: /navigate_oswhite.xml -> text/xml (navigate_oswhite.xml) [159 B]
4: /navigate_bwfix.xml -> text/xml (navigate_bwfix.xml) [157 B]

But I lost to which host they were made and with which method.

thanks
sebas