Traceroute with any packet. Visualize the routes. Discover Middleboxes and Firewalls
Home Page: https://wikicensorship.github.io/
License: The Unlicense
following #36 :
To do better what we did here #22, one solution is to use Socket itself directly. As a result, we do not need to configure the firewall to prevent instant RST packets from being sent by our kernel. Although this problem does not appear to exist in the new Linux kernels (version 5), it may still be worth considering.
An incomplete and slightly wrong code is:
try:
s=socket.socket()
s.settimeout(3)
s.connect(("1.0.0.1",443))
s2=socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_ICMP)
s2.bind(('',0))
s.setsockopt(socket.SOL_IP, socket.IP_TTL, 1)
print(s.getsockname())
print(s.getpeername())
ss=StreamSocket(s,Raw)
ans,unans=ss.sr(TCP(import_hexcap()),timeout=3)
print(ans)
print(unans)
if len(ans) == 0:
data, curr_addr = s2.recvfrom(1500)
IP(data) # won't work here. ;)
print(curr_addr)
finally:
s.setsockopt(socket.SOL_IP, socket.IP_TTL, 68)
s.close()
s2.close()But the main unsolvable problem I saw is that we in Python could not change TCP_RTO_MIN to prevent consecutive retransmissions (every 200 milliseconds). This can increase false positives as well as unpredictable behavior.
some good resources:
Maybe it's good to have a standard data structure like RIPE Atlas.
https://atlas.ripe.net/docs/data_struct/#v5000_traceroute
e.g.
[{
"af": 4,
"dst_addr": "202.12.27.33",
"dst_name": "202.12.27.33",
"endtime": 1637626292,
"from": "151.246.166.238",
"fw": 5020,
"lts": 9,
"msm_id": 5006,
"msm_name": "Traceroute",
"mver": "2.2.1",
"paris_id": 10,
"prb_id": 1000580,
"proto": "UDP",
"result": [{
"hop": 1,
"result": [{
"from": "172.17.0.1",
"rtt": 0.56,
"size": 68,
"ttl": 64
}, {
"from": "172.17.0.1",
"rtt": 0.399,
"size": 68,
"ttl": 64
}, {
"from": "172.17.0.1",
"rtt": 0.391,
"size": 68,
"ttl": 64
}
]
}, {
"hop": 2,
"result": [{
"from": "192.168.10.1",
"rtt": 2.873,
"size": 68,
"ttl": 63
}, {
"from": "192.168.10.1",
"rtt": 2.793,
"size": 68,
"ttl": 63
}, {
"from": "192.168.10.1",
"rtt": 2.778,
"size": 68,
"ttl": 63
}
]
}, {
"hop": 3,
"result": [{
"from": "192.168.1.1",
"ittl": 0,
"rtt": 6.255,
"size": 28,
"ttl": 252
}, {
"from": "192.168.1.1",
"ittl": 0,
"rtt": 3.729,
"size": 28,
"ttl": 252
}, {
"from": "192.168.1.1",
"ittl": 0,
"rtt": 3.408,
"size": 28,
"ttl": 252
}
]
}, {
"hop": 4,
"result": [{
"from": "94.183.140.6",
"rtt": 36.494,
"size": 28,
"ttl": 252
}, {
"from": "94.183.140.6",
"rtt": 34.834,
"size": 28,
"ttl": 252
}, {
"from": "94.183.140.6",
"rtt": 33.892,
"size": 28,
"ttl": 252
}
]
}, {
"hop": 5,
"result": [{
"from": "94.183.140.1",
"rtt": 34.591,
"size": 28,
"ttl": 251
}, {
"from": "94.183.140.1",
"rtt": 34.263,
"size": 28,
"ttl": 251
}, {
"from": "94.183.140.1",
"rtt": 33.309,
"size": 28,
"ttl": 251
}
]
}, {
"hop": 6,
"result": [{
"from": "172.22.133.37",
"rtt": 37.557,
"size": 68,
"ttl": 250
}, {
"from": "172.22.133.37",
"rtt": 37.491,
"size": 68,
"ttl": 250
}, {
"from": "172.22.133.37",
"rtt": 38.068,
"size": 68,
"ttl": 250
}
]
}, {
"hop": 7,
"result": [{
"from": "172.18.218.145",
"rtt": 38.313,
"size": 68,
"ttl": 249
}, {
"from": "172.18.218.137",
"rtt": 38.449,
"size": 68,
"ttl": 249
}, {
"from": "172.18.205.249",
"rtt": 37.496,
"size": 68,
"ttl": 249
}
]
}, {
"hop": 8,
"result": [{
"from": "172.18.188.18",
"rtt": 38.758,
"size": 68,
"ttl": 121
}, {
"from": "172.18.188.18",
"rtt": 38.614,
"size": 68,
"ttl": 121
}, {
"from": "172.18.188.18",
"rtt": 38.552,
"size": 68,
"ttl": 121
}
]
}, {
"hop": 9,
"result": [{
"from": "172.18.188.68",
"rtt": 39.842,
"size": 68,
"ttl": 247
}, {
"from": "172.18.188.68",
"rtt": 39.143,
"size": 68,
"ttl": 247
}, {
"from": "172.18.188.68",
"rtt": 44.919,
"size": 68,
"ttl": 247
}
]
}, {
"hop": 10,
"result": [{
"from": "172.18.192.17",
"rtt": 40.768,
"size": 68,
"ttl": 246
}, {
"from": "172.18.192.25",
"rtt": 39.261,
"size": 68,
"ttl": 246
}, {
"from": "172.18.192.13",
"rtt": 40.284,
"size": 68,
"ttl": 246
}
]
}, {
"hop": 11,
"result": [{
"from": "172.18.181.137",
"rtt": 40.962,
"size": 68,
"ttl": 245
}, {
"from": "172.16.46.1",
"rtt": 41.896,
"size": 68,
"ttl": 245
}, {
"from": "172.16.46.5",
"rtt": 47.197,
"size": 68,
"ttl": 245
}
]
}, {
"hop": 12,
"result": [{
"from": "10.201.177.221",
"rtt": 43.408,
"size": 68,
"ttl": 242
}, {
"from": "10.201.177.225",
"rtt": 43.689,
"size": 68,
"ttl": 242
}, {
"from": "10.201.177.225",
"rtt": 42.649,
"size": 68,
"ttl": 242
}
]
}, {
"hop": 13,
"result": [{
"from": "10.21.21.10",
"rtt": 42.748,
"size": 68,
"ttl": 241
}, {
"from": "10.21.21.10",
"rtt": 41.301,
"size": 68,
"ttl": 241
}, {
"from": "10.21.21.10",
"rtt": 62.887,
"size": 68,
"ttl": 241
}
]
}, {
"hop": 14,
"result": [{
"from": "10.21.211.10",
"icmpext": {
"obj": [{
"class": 0,
"type": 0
}
],
"rfc4884": 1,
"version": 0
},
"ittl": 2,
"rtt": 43.338,
"size": 140,
"ttl": 239
}, {
"from": "10.21.211.10",
"icmpext": {
"obj": [{
"class": 0,
"type": 0
}
],
"rfc4884": 1,
"version": 0
},
"ittl": 2,
"rtt": 45.076,
"size": 140,
"ttl": 239
}, {
"from": "10.21.211.10",
"icmpext": {
"obj": [{
"class": 0,
"type": 0
}
],
"rfc4884": 1,
"version": 0
},
"ittl": 2,
"rtt": 41.775,
"size": 140,
"ttl": 239
}
]
}, {
"hop": 15,
"result": [{
"from": "10.21.21.10",
"rtt": 42.107,
"size": 68,
"ttl": 241
}, {
"from": "10.21.21.10",
"rtt": 46.849,
"size": 68,
"ttl": 241
}, {
"from": "10.21.21.10",
"rtt": 42.075,
"size": 68,
"ttl": 241
}
]
}, {
"hop": 16,
"result": [{
"from": "185.1.48.16",
"rtt": 76.749,
"size": 68,
"ttl": 49
}, {
"from": "185.1.48.16",
"rtt": 74.96,
"size": 68,
"ttl": 49
}, {
"from": "185.1.48.16",
"rtt": 74.57,
"size": 68,
"ttl": 49
}
]
}, {
"hop": 17,
"result": [{
"from": "184.104.196.45",
"rtt": 84.812,
"size": 68,
"ttl": 48
}, {
"from": "184.104.196.45",
"rtt": 82.897,
"size": 68,
"ttl": 48
}, {
"from": "184.104.196.45",
"rtt": 83.57,
"size": 68,
"ttl": 48
}
]
}, {
"hop": 18,
"result": [{
"from": "184.104.196.1",
"rtt": 82.066,
"size": 28,
"ttl": 46
}, {
"from": "184.104.196.1",
"rtt": 82.954,
"size": 28,
"ttl": 46
}, {
"from": "184.104.196.1",
"rtt": 82.795,
"size": 28,
"ttl": 46
}
]
}, {
"hop": 19,
"result": [{
"from": "184.105.65.133",
"rtt": 96.805,
"size": 28,
"ttl": 46
}, {
"from": "184.105.65.133",
"rtt": 96.212,
"size": 28,
"ttl": 46
}, {
"from": "184.105.65.133",
"rtt": 102.439,
"size": 28,
"ttl": 46
}
]
}, {
"hop": 20,
"result": [{
"from": "184.105.65.5",
"rtt": 115.32,
"size": 28,
"ttl": 45
}, {
"from": "184.105.65.5",
"rtt": 111.125,
"size": 28,
"ttl": 45
}, {
"from": "184.105.65.5",
"rtt": 113.071,
"size": 28,
"ttl": 45
}
]
}, {
"hop": 21,
"result": [{
"from": "37.49.237.106",
"rtt": 129.242,
"size": 28,
"ttl": 235
}, {
"from": "195.42.144.79",
"rtt": 130.602,
"size": 28,
"ttl": 235
}, {
"from": "37.49.237.106",
"rtt": 126.654,
"size": 28,
"ttl": 235
}
]
}, {
"hop": 22,
"result": [{
"from": "202.12.27.33",
"rtt": 130.383,
"size": 28,
"ttl": 233
}, {
"from": "202.12.27.33",
"rtt": 126.517,
"size": 28,
"ttl": 233
}, {
"from": "202.12.27.33",
"rtt": 127.824,
"size": 28,
"ttl": 233
}
]
}
],
"size": 40,
"src_addr": "172.17.0.2",
"stored_timestamp": 1637626336,
"timestamp": 1637626288,
"type": "traceroute"
}, {
"af": 4,
"dst_addr": "202.12.27.33",
"dst_name": "202.12.27.33",
"endtime": 1637628093,
"from": "151.246.166.238",
"fw": 5020,
"lts": 14,
"msm_id": 5006,
"msm_name": "Traceroute",
"mver": "2.2.1",
"paris_id": 11,
"prb_id": 1000580,
"proto": "UDP",
"result": [{
"hop": 1,
"result": [{
"from": "172.17.0.1",
"rtt": 0.564,
"size": 68,
"ttl": 64
}, {
"from": "172.17.0.1",
"rtt": 0.399,
"size": 68,
"ttl": 64
}, {
"from": "172.17.0.1",
"rtt": 0.315,
"size": 68,
"ttl": 64
}
]
}, {
"hop": 2,
"result": [{
"from": "192.168.10.1",
"rtt": 2.748,
"size": 68,
"ttl": 63
}, {
"from": "192.168.10.1",
"rtt": 2.916,
"size": 68,
"ttl": 63
}, {
"from": "192.168.10.1",
"rtt": 4.468,
"size": 68,
"ttl": 63
}
]
}, {
"hop": 3,
"result": [{
"from": "192.168.1.1",
"ittl": 0,
"rtt": 3.946,
"size": 28,
"ttl": 252
}, {
"from": "192.168.1.1",
"ittl": 0,
"rtt": 3.641,
"size": 28,
"ttl": 252
}, {
"from": "192.168.1.1",
"ittl": 0,
"rtt": 3.745,
"size": 28,
"ttl": 252
}
]
}, {
"hop": 4,
"result": [{
"from": "94.183.140.6",
"rtt": 34.607,
"size": 28,
"ttl": 252
}, {
"from": "94.183.140.6",
"rtt": 34.368,
"size": 28,
"ttl": 252
}, {
"from": "94.183.140.6",
"rtt": 33.883,
"size": 28,
"ttl": 252
}
]
}, {
"hop": 5,
"result": [{
"from": "94.183.140.1",
"rtt": 35.126,
"size": 28,
"ttl": 251
}, {
"from": "94.183.140.1",
"rtt": 34.16,
"size": 28,
"ttl": 251
}, {
"from": "94.183.140.1",
"rtt": 34.08,
"size": 28,
"ttl": 251
}
]
}, {
"hop": 6,
"result": [{
"from": "172.22.133.37",
"rtt": 40.524,
"size": 68,
"ttl": 250
}, {
"from": "172.22.133.37",
"rtt": 37.733,
"size": 68,
"ttl": 250
}, {
"from": "172.22.133.37",
"rtt": 37.492,
"size": 68,
"ttl": 250
}
]
}, {
"hop": 7,
"result": [{
"from": "172.18.218.137",
"rtt": 38.053,
"size": 68,
"ttl": 249
}, {
"from": "172.18.205.213",
"rtt": 38.36,
"size": 68,
"ttl": 249
}, {
"from": "172.18.222.209",
"rtt": 37.867,
"size": 68,
"ttl": 249
}
]
}, {
"hop": 8,
"result": [{
"from": "172.18.188.18",
"rtt": 37.903,
"size": 68,
"ttl": 121
}, {
"from": "172.18.188.18",
"rtt": 39.561,
"size": 68,
"ttl": 121
}, {
"from": "172.18.188.18",
"rtt": 37.919,
"size": 68,
"ttl": 121
}
]
}, {
"hop": 9,
"result": [{
"from": "172.18.188.68",
"rtt": 41.651,
"size": 68,
"ttl": 247
}, {
"from": "172.18.188.68",
"rtt": 39.315,
"size": 68,
"ttl": 247
}, {
"from": "172.18.188.68",
"rtt": 40.564,
"size": 68,
"ttl": 247
}
]
}, {
"hop": 10,
"result": [{
"from": "172.18.192.1",
"rtt": 38.432,
"size": 68,
"ttl": 246
}, {
"from": "172.18.192.29",
"rtt": 39.749,
"size": 68,
"ttl": 246
}, {
"from": "172.18.192.5",
"rtt": 53.008,
"size": 68,
"ttl": 246
}
]
}, {
"hop": 11,
"result": [{
"from": "172.16.46.9",
"rtt": 41.886,
"size": 68,
"ttl": 245
}, {
"from": "172.16.46.193",
"rtt": 41.042,
"size": 68,
"ttl": 245
}, {
"from": "172.16.46.197",
"rtt": 42.388,
"size": 68,
"ttl": 245
}
]
}, {
"hop": 12,
"result": [{
"from": "10.201.177.225",
"rtt": 41.305,
"size": 68,
"ttl": 242
}, {
"from": "10.201.177.221",
"rtt": 44.863,
"size": 68,
"ttl": 242
}, {
"from": "10.201.177.225",
"rtt": 47.7,
"size": 68,
"ttl": 242
}
]
}, {
"hop": 13,
"result": [{
"from": "10.21.21.10",
"rtt": 42.65,
"size": 68,
"ttl": 241
}, {
"from": "10.21.21.10",
"rtt": 42.842,
"size": 68,
"ttl": 241
}, {
"from": "10.21.21.10",
"rtt": 42.72,
"size": 68,
"ttl": 241
}
]
}, {
"hop": 14,
"result": [{
"from": "10.21.211.10",
"icmpext": {
"obj": [{
"class": 0,
"type": 0
}
],
"rfc4884": 1,
"version": 0
},
"ittl": 2,
"rtt": 43.051,
"size": 140,
"ttl": 239
}, {
"from": "10.21.211.10",
"icmpext": {
"obj": [{
"class": 0,
"type": 0
}
],
"rfc4884": 1,
"version": 0
},
"ittl": 2,
"rtt": 43.543,
"size": 140,
"ttl": 239
}, {
"from": "10.21.211.10",
"icmpext": {
"obj": [{
"class": 0,
"type": 0
}
],
"rfc4884": 1,
"version": 0
},
"ittl": 2,
"rtt": 43.603,
"size": 140,
"ttl": 239
}
]
}, {
"hop": 15,
"result": [{
"from": "10.21.21.10",
"rtt": 42.254,
"size": 68,
"ttl": 241
}, {
"from": "10.21.21.10",
"rtt": 44.902,
"size": 68,
"ttl": 241
}, {
"from": "10.21.21.10",
"rtt": 41.691,
"size": 68,
"ttl": 241
}
]
}, {
"hop": 16,
"result": [{
"from": "185.1.48.16",
"rtt": 77.511,
"size": 68,
"ttl": 49
}, {
"from": "185.1.48.16",
"rtt": 74.563,
"size": 68,
"ttl": 49
}, {
"from": "185.1.48.16",
"rtt": 76.154,
"size": 68,
"ttl": 49
}
]
}, {
"hop": 17,
"result": [{
"from": "184.104.196.45",
"rtt": 84.355,
"size": 68,
"ttl": 48
}, {
"from": "184.104.196.45",
"rtt": 85.82,
"size": 68,
"ttl": 48
}, {
"from": "184.104.196.45",
"rtt": 82.819,
"size": 68,
"ttl": 48
}
]
}, {
"hop": 18,
"result": [{
"from": "184.104.196.1",
"rtt": 84.804,
"size": 28,
"ttl": 46
}, {
"from": "184.104.196.1",
"rtt": 83.745,
"size": 28,
"ttl": 46
}, {
"from": "184.104.196.1",
"rtt": 81.614,
"size": 28,
"ttl": 46
}
]
}, {
"hop": 19,
"result": [{
"from": "184.105.65.133",
"rtt": 96.806,
"size": 28,
"ttl": 46
}, {
"from": "184.105.65.133",
"rtt": 95.489,
"size": 28,
"ttl": 46
}, {
"from": "184.105.65.133",
"rtt": 96.774,
"size": 28,
"ttl": 46
}
]
}, {
"hop": 20,
"result": [{
"from": "184.105.65.5",
"rtt": 119.542,
"size": 28,
"ttl": 45
}, {
"from": "184.105.65.5",
"rtt": 111.951,
"size": 28,
"ttl": 45
}, {
"from": "184.105.65.5",
"rtt": 113.013,
"size": 28,
"ttl": 45
}
]
}, {
"hop": 21,
"result": [{
"from": "195.42.144.79",
"rtt": 138.066,
"size": 28,
"ttl": 235
}, {
"from": "37.49.237.106",
"rtt": 128.829,
"size": 28,
"ttl": 235
}, {
"from": "37.49.237.106",
"rtt": 132.527,
"size": 28,
"ttl": 235
}
]
}, {
"hop": 22,
"result": [{
"from": "202.12.27.33",
"rtt": 129.301,
"size": 28,
"ttl": 233
}, {
"from": "202.12.27.33",
"rtt": 124.837,
"size": 28,
"ttl": 233
}, {
"from": "202.12.27.33",
"rtt": 125.361,
"size": 28,
"ttl": 233
}
]
}
],
"size": 40,
"src_addr": "172.17.0.2",
"stored_timestamp": 1637628137,
"timestamp": 1637628089,
"type": "traceroute"
}]... it could be beneficial to also present the results in a tabular format, to make it more obvious which results are pertaining to which target IP (I was initially confused because I misread some of the IP targets as being the same, while they were not)
Suggestion: with --tabular argument and input should be a json file path.
I think there would be no problem to do both IPv6 and IPv4 tests at the same time. But since the routes are completely different, how do we visualize them? In two separate files?
Today I tried to run tracevis with a specific target ( with the domain name and "-i" parameter"), and everything went smooth until saving the measurement graph, in which I got a traceback
Traceback (most recent call last): File "./tracevis.py", line 342, in <module> main(get_args(sys.argv[1:])) File "./tracevis.py", line 335, in main if utils.vis.vis( File "C:\Users\Kouros\desktop\tracevis\utils\vis.py", line 277, in vis dst_addr_id = 'x' + str(int(ipaddress.IPv4Address(dst_addr))) + 'x' File "C:\Users\Kouros\AppData\Local\Programs\Python\Python38\lib\ipaddress.py", line 1252, in __init__ self._ip = self._ip_int_from_string(addr_str) File "C:\Users\Kouros\AppData\Local\Programs\Python\Python38\lib\ipaddress.py", line 1144, in _ip_int_from_string raise AddressValueError("Expected 4 octets in %r" % ip_str) ipaddress.AddressValueError: Expected 4 octets in 'TARGET.DOMAIN.DOMAIN'
Line 45 in 63bc2df
(maybe square)
It would be great if we could run tests related to each request with OONI Probe as well.
But how? 🤔
following: #42
reference: https://datatracker.ietf.org/doc/html/rfc3135#section-3.1
Can we help them or we should just learn from them?
Currently, we only have hopbyhop probing algorithm
reference: https://manpages.ubuntu.com/manpages/trusty/man8/paris-traceroute.8.html
حسین(و حسینها) این روزها در زندان میباشند. به این دلیل که برای آزادی و آگاهی گام برداشته اند. حسین (و حسینها) را آزاد کنید.
Hossein has been imprisoned for the sake of "raising the awareness", free Hossein
following: #71 (comment)
to reproduce:
start the test like this in linux without internet:
./tracevis.py --dns -i "1.1.1.1" -m 1 -n "somethin"
then connect to internet before it begins to trace
it will print the meta information after the trace:
One solution could be to use an HTML template.
It will be very useful and important to have reverse trace route. We just need to receive the client IP address from an argument and then respond to the incoming packet in trace way. (i.e, reverse Paris-traceroute)
the next great step after this would be to do Paris-traceroute on both side at the same time. then with #31 we can have a very interesting/useful data and graph.
reference: https://research.cs.washington.edu/networking/astronomy/reverse-traceroute.html
follow-up of #62
maybe we need:
In the study, I came across an article that described a fingerprint I accidentally found. (Identification based on TTL mirroring)
This article also mentions two other fingerprints. Which would be great if we could implement it. This is because the article mentions that these middleboxes are used in parallel in China.
https://www.usenix.org/conference/foci20/presentation/anonymous
there's a lack of tcp-ttl-ping-with-data utility to detect the middleboxes.
for example we have a DPI censorship which is triggered only with certain HTTP/TLS request. The program need to establish TCP session SYN, SYN/ACK, ACK, and then send a HTTP/TLS data packet gradually increasing TTL until the censorship packet received.
we should have a better readme
The order should be in this way so that the same colors are for the same file:
resultA[0]
resultB[0]
resultA[1]
resultB[1]
resultA[2]
resultB[2]Have an interactive environment to generate a packet with Scapy and perform operations with utils/trace.py
For now, users can do this to have hexdump of a generated packet:
or just hexdump(IP()/UDP()/DNS())
follow-up of #55
There is a common mistake and that is that users make a lot of mistakes in providing the correct packet.payload. We should cover their mistakes and/or show the appropriate error if possible.
For example, if the hexdump didn't start with '4' (i.e, IPv4 -- currently supported), show an error. Or try Ether(hexdump) instead of IP(hexdump), and see if there is an IP layer, export packet.payload or packet[IP] to remove the Ethernet layer
We should be able to anonymize a json data.
Initial suggestion:
python3 ./tracevis.py -f ./path/to/dir/data.json --anonymize
Then:
user_possible_IP= src_addr
s/user_possible_IP/127.1.2.7/g
Output: ./path/to/dir/data_anonymized.json
Maybe we should also change ip.src of packet before saving in .conf file. (there is no problem in doing this)
Anything else?
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.