hi sir, when I run this code:$ cd " clone directory "
the ubunto said = syntax error near unexpected token `newline'
what's the problem? ' i have done all of the step i mean download that code.

used Version: 1.4
used mrt files: samples/quagga_bgp, samples/quagga_rib, one from RIPE RIS (updates dump), two from routeviews.org (updates dump, rib dump)

observed behaviour:
with (-A|-G) flag (api (grouping) format): mrt2exabgp.py doesn't finish by itself; it prints the correct routes with rib dumps (nothing for updates dumps->other issue) but it stays alive till the user finishes it with strg+c for example

Regards,
Daniel

Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\salmacmp\AppData\Local\Temp\pip-req-build-fs_0nxwc\setup.py", line 32, in
import mrtparse
File "C:\Users\salmacmp\AppData\Local\Temp\pip-req-build-fs_0nxwc\mrtparse_init_.py", line 31, in
signal.signal(signal.SIGPIPE, signal.SIG_DFL)
AttributeError: module 'signal' has no attribute 'SIGPIPE'

Hi there,
In the init.py file there's a typo on line 930 when path_seg['len'] is called. I believe it should be path_seg['length'] instead, as there is no attribute len in a BGP message.

I was trying to use mrtparse for exabgp in our lab and found that the prefix length key is set to just length, where in the json dump the key is prefix_length. I can submit a PR if you like.

(.venv) [jhall@fooji bgp]$ pip list | grep mrt
mrtparse 2.1.0

Hi, I have been able to advertise the full internet table with exabgp using the old 'announce route' syntax however I believe there is a more efficient way to do this using a newer 'api' format. I can see in the article that there is an 'announce attributes' syntax. https://github.com/Exa-Networks/exabgp/wiki/Large-Configuration-File and that mrtparse allows you to generate this config using the flags -G and -P

I have used the command:

python /usr/local/lib/python2.7/dist-packages/mrtparse/examples/mrt2exabgp.py -G -P latest-bview.gz  > fullbgptable.py

When I run this command the process hangs and does that send any output to the file. If I omit the -G then the command works but I end up with the old syntax.

If anyone has any experience with getting this working then any help would be appreciated.

Thanks.

Hey!

It would be nice to be able to do "pip install mrtparse".

you seem to create the function val_dict only for the reason to return a default if the value is not in the dict. Without the default value the following lines are equivalent:

>>> MSG_ST[9][4]
'BGP_SYNC'
>>> val_dict(MSG_ST, 9, 4)
'BGP_SYNC'

This is a bit unpythonic and the actual meaning of the function is a bit hided. There are two better solutions for val_dict:

First assign the default value direct on accessing content. This has the advantage that the user see that it returns a default on Key Error, but it is more to write and not as nice as MSG_ST[9][4]:

>>> MSG_ST.get(9, dict()).get(4, 'unknown')
'BGP_SYNC'
>>> MSG_ST.get(9, dict()).get(90, 'unknown')
'unknown'
>>> MSG_ST.get(90, dict()).get(4, 'unknown')
'unknown'

If you want that behaviour in any case the best, nicest and cleanest solution would be to simple subclass the dict class:

>>> import collections
>>> BGP_ST = collections.defaultdict(lambda: "unknown", BGP_ST)
>>> BGP_ST[4]
'BGP_SYNC'
>>> BGP_ST[90]                                                                                                 
'unknown'

for dicts which returns dicts you have to use:

>>> MSG_ST = collections.defaultdict(lambda: dict(), MSG_ST)
>>> MSG_ST[9]                                                                                                          
{0: 'BGP_NULL', 1: 'BGP_UPDATE', 2: 'BGP_PREF_UPDATE', 3: 'BGP_STATE_CHANGE', 4: 'BGP_SYNC', 5: 'BGP_OPEN', 6: 'BGP_NOTIFY', 7: 'BGP_KEEPALIVE', 'BGP_KEEPALIVE': 7, 'BGP_NOTIFY': 6, 'BGP_NULL': 0, 'BGP_OPEN': 5, 'BGP_SYNC': 4, 'BGP_STATE_CHANGE': 3, 'BGP_UPDATE': 1, 'BGP_PREF_UPDATE': 2}
>>> MSG_ST[90]
{}

Of course the dics gets anyway obsolete with #5

Hi,

I used print_all.py to print the output of a MRT file from RIPE RouteCollector rrc21 (http://data.ris.ripe.net/rrc21/2020.04/updates.20200422.0125.gz). It seems that IPv4 prefixes at the end of BGP message are getting decoded as IPv6. For example, following is an example taken from the output of print_all.py:

MRT Header

 Timestamp: 1587518739(2020-04-22 01:25:39)
  Type: 16(BGP4MP)
  Subtype: 4(BGP4MP_MESSAGE_AS4)
  Length: 267
BGP4MP_MESSAGE_AS4
  Peer AS Number: 42275
  Local AS Number: 12654
  Interface Index: 0
  Address Family: 2(IPv6)
  Peer IP Address: 2001:7f8:54::1:204
  Local IP Address: 2001:7f8:54::1:99
BGP Message
  Marker: -- ignored --
  Length: 223
  Type: 2(UPDATE)
  Withdrawn Routes Length: 0
  Total Path Attribute Length: 111
  Path Attribute Flags/Type/Length: 0x40/1/1
      ORIGIN: 0(IGP)
  Path Attribute Flags/Type/Length: 0x40/2/30
      AS_PATH
          Path Segment Type: 2(AS_SEQUENCE)
          Path Segment Length: 7
          Path Segment Value: 42275 25091 57463 263444 6762 16509 14618
  Path Attribute Flags/Type/Length: 0x40/3/4
      NEXT_HOP: 85.208.68.1
  Path Attribute Flags/Type/Length: 0xc0/8/64
      COMMUNITY: 0:8304 0:49981 16:1134 6762:1 6762:31 6762:40 6762:10150 24115:57463 25091:22414 25091:24115 64700:57463 64704:1300 65400:0 65400:65400 65444:102 65444:2000
  NLRI: 305::/20
  NLRI: 305::/24
  NLRI: 305:1000::/21
  NLRI: 305:1000::/24
  NLRI: 350::/12
  NLRI: 3e0::/12
  NLRI: 3d0::/12
  NLRI: 305:400::/24
  NLRI: 305:900::/24
  NLRI: 305:1100::/24
  NLRI: 305:1200::/24
  NLRI: 305:a00::/24
  NLRI: 305:b00::/24
  NLRI: 305:700::/24
  NLRI: 305:1300::/24
  NLRI: 305:200::/24
  NLRI: 305:600::/24
  NLRI: 305:300::/24
  NLRI: 305:1500::/24
  NLRI: 305:500::/24
  NLRI: 305:800::/24
  NLRI: 305:100::/24
  NLRI: 305:1400::/24

It seems that the parser is using the AFI (IPv6) provided in the BGP4MP_MESSAGE header to decode the prefixes at the end of the BGP message whereas RFC 6396 under 4.4.2 states that:

The Address Family value only applies to the IP addresses contained
in the MRT header. The BGP4MP_MESSAGE Subtype is otherwise
transparent to the contents of the actual message that may contain
any valid AFI/SAFI values. Only one BGP message SHALL be encoded in
the BGP4MP_MESSAGE Subtype.

In https://github.com/t2mune/mrtparse/blob/master/examples/mrt2bgpdump.py

I believe there is an issue on line 285:
seg_t = list(seg['type'])[0]
for seg in attr['value']:
if seg_t == AS_PATH_SEG_T['AS_SET']:

I believe it should be
for seg in attr['value']:
seg_t = list(seg['type'])[0]
if seg_t == AS_PATH_SEG_T['AS_SET']:

As in line 242 since seg does not exist prior.

Hello, Thank you very much for writing this very useful tool.

I noticed that exabgp_conf.py can be generate very large configuration files which will most often will then take ages to load. A faster way to do would be to send the routes using the API. The process has been documented by another ExaBGP user here: https://github.com/FastVPSEestiOu/fastnetmon/blob/master/docs/EXABGP_INTEGRATION.md
or more elegantly https://github.com/job/irrexplorer/blob/master/exabgp.conf

Also, I added an "undocumented" (i.e.: needing some improvement but working) command to the API allowing to reduce parsing by grouping updates with the same attributes. From memory it is something like "announce attribute ... route ... ". I would need to double check if this is of interest.

Good continuation.

used Version: 1.4
used mrt files: samples/quagga_bgp, one from RIPE RIS (updates dump), one from routeviews.org (updates dump)

observed behaviour:
without any flag (config format): prints the config header and footer but no static routes

with (-A|-G) flag (api (grouping) format): prints nothing (and doesn't finish by itself; strg+c at 0% CPU over long time->other issue)

my guess:
It seems like mrt2exabgp.py can't extract routes from the BGP4MP_MESSAGE type.

Regards,
Daniel

Hello , I wish install full routing table in a Router but before Convert the RAW BGP data from RIPE into a format that ExaBGP can process, however i get a issue.

I tried with mrtparse and http://data.ris.ripe.net/rrc16/latest-bview.gz and then with mrt2exabgp -G -P ./latest-bview.gz > fullbgptable.py. The session is UP, not received prefix, but i got the same error message in the router.

sh bgp summary
Neighbor Spk AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down St/PfxRcd
192.168.1.111 0 200 1608 12 4 0 0 00:00:24 0

RP/0/0/CPU0:ios#RP/0/0/CPU0:Oct 21 02:05:15.424 : bgp[1052]: %ROUTING-BGP-3-MALFORM_UPDATE : Malformed UPDATE message received from neighbor 192.168.1.111 (VRF: default) - message length 269 bytes, error flags 0x00000200, action taken "TreatAsWdr". Error details: "Error 0x00000200, Field "Attr-data", Attribute 2 (Flags 0x40, Length 18), Data [40021202040000cc6000000da300000d]". NLRIs: [IPv4 Unicast] 217.14.120.0/22 205.167.22.0/23 205.167.188.0/23 206.221.126.0/23 199.254.216.0/24 200.41.49.0/24 200.41.53.0/24...

The code after mrtparse is:

#!/usr/bin/env python

import sys
import time

msgs = [
'announce attributes origin IGP as-path [199524 2914 ] next-hop 198.32.243.133 nlri 0.0.0.0/0',
'announce attributes origin IGP as-path [199524 2914 ] next-hop 2001:504:0:6:0:19:9524:1 nlri ::/0 2001:218::/32 2001:218:2002::/48 2001:218:8000::/38 2001:418::/32 2001:728::/32 2>
'announce attributes origin IGP as-path [52320 13335 ] med 0 aggregator (13335:108.162.211.10) community [52320:31912] next-hop 198.32.242.254 nlri 1.0.0.0/24 1.1.1.0/24 8.27.64.0/>
'announce attributes origin IGP as-path [6939 4826 38803 ] med 0 next-hop 198.32.242.176 nlri 1.0.4.0/22 1.0.4.0/24 1.0.5.0/24 1.0.6.0/24 1.0.7.0/24 43.252.108.0/22 43.252.108.0/24>
'announce attributes origin IGP as-path [52320 3491 2516 7670 18144 ] med 0 aggregator (18144:219.118.225.18) community [52320:11912] next-hop 198.32.242.254 nlri 1.0.64.0/18 58.18>
'announce attributes origin IGP as-path [6939 38040 23969 ] med 0 next-hop 198.32.242.176 nlri 1.0.128.0/17 1.0.128.0/18 1.0.128.0/19 1.0.130.0/23 1.0.132.0/24 1.0.133.0/24 1.0.136>
'announce attributes origin IGP as-path [6939 4651 23969 ] med 0 next-hop 198.32.242.176 nlri 1.0.128.0/24 1.0.129.0/24 1.0.137.0/24 1.0.144.0/20 1.0.160.0/22 1.0.164.0/24 1.0.165.>
'announce attributes origin IGP as-path [52320 4134 58543 ] med 0 community [52320:21912] next-hop 198.32.242.254 nlri 1.1.8.0/24 121.201.0.0/18 121.201.64.0/18',
.....
while msgs:
msg = msgs.pop(0)
if isinstance(msg, str):
sys.stdout.write(msg + '\n')
sys.stdout.flush()
else:
time.sleep(msg)

while True:
time.sleep(1)

How can i get data in correct format ?

The "announce attribute ..." syntax of the API grouping format was changed a little. It has to be "...attributes..." with s now in the current version.

Regards,
Daniel

Using http://data.ris.ripe.net/rrc00/latest-bview.gz:

root@pgen:/usr/local/etc/exabgp# /home/jlixfeld/mrtparse/examples/mrt2exabgp.py -A /home/jlixfeld/latest-bview.gz
announce route 0.0.0.0/0 origin IGP as-path [45896 3356 ] community [3356:3 3356:9 3356:575 3356:2003] next-hop 111.91.233.1
announce route ::/0 origin IGP as-path [29608 5511 5511 5511 5511 36925 ] med 11 community [5511:500 5511:518 5511:999 5511:5511 29608:30400] next-hop 2a01:678::2
announce route 1.0.4.0/22 origin IGP as-path [45896 4826 38803 56203 ] community [19996:2001] next-hop 111.91.233.1
announce route 1.0.4.0/24 origin IGP as-path [45896 4826 38803 56203 ] community [19996:2001] next-hop 111.91.233.1
...
...
...
Traceback (most recent call last):
File "/home/jlixfeld/mrtparse/examples/mrt2exabgp.py", line 498, in
main()
File "/home/jlixfeld/mrtparse/examples/mrt2exabgp.py", line 495, in main
conv_format(args, flags, d)
File "/home/jlixfeld/mrtparse/examples/mrt2exabgp.py", line 191, in conv_format
for m in d:
File "/usr/local/lib/python2.7/dist-packages/mrtparse-1.6-py2.7.egg/mrtparse/init.py", line 609, in next
self.unpack_data()
File "/usr/local/lib/python2.7/dist-packages/mrtparse-1.6-py2.7.egg/mrtparse/init.py", line 640, in unpack_data
data = self.f.read(self.mrt.len)
File "/usr/lib/python2.7/gzip.py", line 268, in read
self._read(readsize)
File "/usr/lib/python2.7/gzip.py", line 315, in _read
self._read_eof()
File "/usr/lib/python2.7/gzip.py", line 354, in _read_eof
hex(self.crc)))
IOError: CRC check failed 0x579c0579 != 0xd1fd5a7dL

I'm not well enough versed in Python to really be able to understand how to troubleshoot this. Any assistance would be appreciated!

Hello Dear All,

When I run below command, I am receiving error.
Could you please help me?

[root@centos examples]# time python3 mrt2exabgp.py -G -P latest-bview.gz > fullbgptable.py
^CTraceback (most recent call last):
File "mrt2exabgp.py", line 514, in
main()
File "mrt2exabgp.py", line 511, in main
conv_format(args, flags, d)
File "mrt2exabgp.py", line 179, in conv_format
for m in d:
File "/usr/local/lib/python3.6/site-packages/mrtparse/init.py", line 94, in next
self.unpack_msg(mrt)
File "/usr/local/lib/python3.6/site-packages/mrtparse/init.py", line 138, in unpack_msg
self.unpack_td_v2(buf, mrt)
File "/usr/local/lib/python3.6/site-packages/mrtparse/init.py", line 185, in unpack_td_v2
rib.unpack()
File "/usr/local/lib/python3.6/site-packages/mrtparse/init.py", line 381, in unpack
self.p += entry.unpack()
File "/usr/local/lib/python3.6/site-packages/mrtparse/init.py", line 410, in unpack
self.p += attr.unpack()
File "/usr/local/lib/python3.6/site-packages/mrtparse/init.py", line 703, in unpack
self.unpack_next_hop()
File "/usr/local/lib/python3.6/site-packages/mrtparse/init.py", line 766, in unpack_next_hop
self.data['value'] = self.val_addr(AFI_T['IPv4'])
File "/usr/local/lib/python3.6/site-packages/mrtparse/base.py", line 301, in val_addr
addr = socket.inet_ntop(_af, buf + b'\x00'*(plen_max // 8 - n))
KeyboardInterrupt

real 0m5.786s
user 0m5.513s
sys 0m0.058s
[root@centos examples]#

I must be doing something wrong, I'm using the latest git checkout.

I am trying to pull out all the Cogent routing information for AS174...

wget http://data.ris.ripe.net/rrc00/2020.11/bview.20201101.0000.gz
examples/mrt2exabgp.py -p 174 -s -A bview.20201111.0000.gz

The error I'm getting is this...

$ examples/mrt2exabgp.py -p 174 -s -A bview.20201111.0000.gz
Traceback (most recent call last):
  File "examples/mrt2exabgp.py", line 514, in <module>
    main()
  File "examples/mrt2exabgp.py", line 511, in main
    conv_format(args, flags, d)
  File "examples/mrt2exabgp.py", line 185, in conv_format
    print_route_td(args, params, m.data)
  File "examples/mrt2exabgp.py", line 250, in print_route_td
    line += get_bgp_attr(args, params, m, attr)
  File "examples/mrt2exabgp.py", line 397, in get_bgp_attr
    path_seg = attr['as_path'][0]
KeyError: 'as_path'

Any help would be greatly appreciated!

I'm running print_all.py for a Route Views update file and am getting the error below

File "/Users/rodrigodalri/Downloads/BGPstability/mrtparse/init.py", line 454, in chk_buf
if len(self.buf) - self.p < n:
TypeError: object of type 'NoneType' has no len()

Any idea what that might be?

Bird already supports Extended NLRI Encodings in order to offer Advertisement of Multiple Paths in BGP. When bird is configured with the multiple paths, the nlris are extended encoded on the mrt path.

Is it possible to add the support of extended nlri encodings?

https://tools.ietf.org/html/draft-ietf-idr-add-paths-10

Hi,

First of all I want to say that I appreciate a lot this tool and that it's been helping me to read the BGP Updates messages. Nevertheless, I am having some trouble now. Here is what happens:

1. I am downloading one of the updates from: http://routeviews.org/bgpdata/2018.03/UPDATES/
   For example: updates.20180322.1430.bz2

2. I run: python2.7 mrt2bgpdump.py -m -O output.txt updates.20180322.1430.bz2
   And it just crashes with the following output:

Traceback (most recent call last):
File "mrt2bgpdump.py", line 302, in
main()
File "mrt2bgpdump.py", line 298, in main
b.bgp4mp(m, count)
File "mrt2bgpdump.py", line 206, in bgp4mp
self.bgp_attr(attr)
File "mrt2bgpdump.py", line 268, in bgp_attr
self.as4_aggr = '%d %s' % (attr.as4_aggr['asn'], attr.as4_aggr['ip'])
KeyError: 'ip'

In line 267 and 268 we have:
elif attr.type == BGP_ATTR_T['AS4_AGGREGATOR']:
self.as4_aggr = '%d %s' % (attr.as4_aggr['asn'], attr.as4_aggr['ip'])

So for some reason the dictionary is not recognizing the key.

It seems that in all the previous tables I was using this attribute doesn't appear (at least I checked a couple and effectively it's not there). But now that I am taking Updates from routeviews, it creates some trouble. I commented the line after the if, and with a print I could see that the cases in which this attribute appears is quite low, but I did not know what else I could do, so I just leave you this message.

I hope I am not making any horrible mistake!

Thanks for your time in advance,

Best regards,
Julian.