I was wondering if you could help me in understanding the wiki page for iprange.
I've been trying to understand how to best reduce my ip sets. From what I can tell an ipset can be millions of entries long and there will be no appreciable loss in speed. The only trade off is in ram usage. The larger the set the greater the ram. Do you know if that is really the case?

I have a raw country.list that is 19193 entries long, with 19 CIDRs, and takes up .48MB or ram by running
ipset list country.list
If I run iprange like this:
iprange -v --ipset-reduce 0 --ipset-reduce-entries 100000 country.list >/dev/null
I get a set that contains:
3 CIDR prefixes
64012 entries
Ram now nearing 1.5MB

I have other sets using as much as 23MB of ram, 26 CIDR prefixes, and over 90000 entries. I could reduce them further but they are going to get tremendously huge.
Do you have any thoughts on if this will slow the network down or if the only adverse effect is ram consumption?
Is there an amount of ram I should be trying to keep my sets under?

Thanks for creating such a great tool.

Hi there, question, is it possible to somehow specify the DNS server which iprange should use to resolve hostnames to IPs? I'd like it to use a DNS server different from the one configured in the host if possible...

Thanks.

I have downloaded the tar file file from github.
Then after unzipping it, I am facing problem in make && make install

Here is the error,

make all-am
make[1]: Entering directory /home/luvpreet/Downloads/iprange-1.0.3' make[1]: Leaving directory /home/luvpreet/Downloads/iprange-1.0.3'
make[1]: Entering directory /home/luvpreet/Downloads/iprange-1.0.3' /bin/mkdir -p '/usr/local/bin' /usr/bin/install -c iprange '/usr/local/bin' /bin/mkdir -p '/usr/local/share/man/man1' /usr/bin/install -c -m 644 iprange.1 '/usr/local/share/man/man1' make[1]: Leaving directory /home/luvpreet/Downloads/iprange-1.0.3'

My question is, was it a typo or there is a release missing?

This isn't directly mentioned on the wiki page, but I had a hunch and after experimenting I found out that iprange will read numeric notation directly! So something like:

iprange 18279424 -> 1.22.236.0
iprange 18279424/24 -> 1.22.236.0/24
iprange 18279424-18279679 -> 1.22.236.0/24

Pretty nifty! That's less conversion back & forth when storing IPs in numeric / integer form.

When I use reduce with unsorted ip list, I get strange errors. (Data for example.list are bellow.)

iprange --reduce-factor 20 example.list

iprange: WARNING: invalid range reversed start=x.x.x.80 end=0.0.x.x
iprange: WARNING: invalid range reversed start=x.x.x.96 end=0.0.x.x
iprange: WARNING: invalid range reversed start=255.255.255.255 end=0.0.0.0
iprange: WARNING: invalid range reversed start=x.x.x.160 end=0.0.127.x

x seems random.

But when I provide same data into stdin, it works without error:

cat example.list | iprange --reduce-factor 20

It also works if I optimize data first:

iprange -J example.list > example2.list
iprange --reduce-factor 20 example2.list

I'm using version 1.0.3+ds-1 from debian stretch, but i'm on debian jessie, so it's possible it is somehow related.

cat > example.list <<EOF
216.144.250.150
69.162.124.226
69.162.124.227
69.162.124.228
69.162.124.229
69.162.124.230
69.162.124.231
69.162.124.232
69.162.124.233
69.162.124.234
69.162.124.235
69.162.124.236
69.162.124.237
63.143.42.242
63.143.42.243
63.143.42.244
63.143.42.245
63.143.42.246
63.143.42.247
63.143.42.248
63.143.42.249
63.143.42.250
63.143.42.251
63.143.42.252
63.143.42.253
46.137.190.132
122.248.234.23
188.226.183.141
178.62.52.237
54.79.28.129
54.94.142.218
104.131.107.63
54.67.10.127
54.64.67.106
159.203.30.41
46.101.250.135
18.221.56.27
EOF

I am getting this error below when I use an IP set in my firehol.conf.
I have iprange, firehol, and netdata all installed.
I am do not have "update-ipsets" installed.
I am using only the firehol_level1.netset.

ipset4 create firehol_level1 hash:net
ipset4 addfile firehol_level1 ipsets/firehol_level1.netset
blacklist4 full ipset:firehol_level1

WARNING 16@/etc/firehol/firehol.conf: blacklist4:: iprange command is not installed - ipsets will not be optimal.

I have iprange installed:

[root@host]# which iprange
/usr/sbin/iprange

It works fine on the command line.

As a workaround, I run the ipset-apply.sh script on the firehol_level1 netset, after I run "firehol start"

The way I understand it, it is best to run iprange on the raw firehol_level.netset file to optimize it?
Is that what it is trying to do when I run "firehol start" and I have an ipset in the config?

Should I manually optimize it to a new file for firehol.conf to use?
Is there a path I can specify in firehol.conf so it can find iprange?

Thanks in advance.
It's all working great, I just want to make sure I am using it most efficiently for production servers.

Please add an option to configure the output of DNS resolution.
For example, I'd like to see which domains can't be resolved (e.g. don't exist):

github.com           140.82.118.4
dead-domain.net

By default, the program only writes A records of selected domains.

@alonbl iprange should be installed in /usr/bin, not /usr/sbin. There is no reason for it to be admin only. Anyone should be able to use it.

@alonbl @philwhineray

Can we release a new minor version that fixes an issue with DNS resolution?

This is the commit: c95f44e

A line was missing.

The man page of iprage says the option --ipset-reduce has "the internal default PERCENT is 20". Yet, when you don't use a number after the option strange things happen.

For instance:

iprange -v --ipset-reduce input.set > output.set

Will sit there and wait saying:

iprange: Loading from stdin

If you move the -v after the reduce:

iprange --ipset-reduce -v input.set > output.set

It appears to work, but it does NOT output any of the verbose content that it should.

Only way to get it to work as expected is to include the percentage amount, i.e.:

iprange -v --ipset-reduce 20 input.set > output.set

If you run without the -v or percent, it just sits there (presumably waiting for stdin).

iprange --ipset-reduce input.set > output.set

So I'm not sure if it's just the documentation not being real specific (what would be point be of saying what an internal default is if you must explicitly always specify it?) or an actual bug.

@alonbl if the user does:

make CFLAGS="-O3"

then it does not compile. You have to remember to do:

make CFLAGS="-O3 -pthreads"

Can we move -pthreads off the CFLAGS to allow the user overwrite optimizations?

This is probably an edge case, but I figured still worth reporting...

I'm attaching a file combined.txt

which is a concatenation of the cymru full bogons & cidr-report bogons lists.

When you run the command iprange -v combined.txt > combined.ipset.txt

It gives the warning: iprange: WARNING: invalid range reversed start=224.0.0.0 end=0.255.255.255

.. and the end file combined.ipset.txt has loads of duplicate IPs (CIDRs) in it. If you also look at line 3073 you will see it wrote 224.0.0.0 with no CIDR. So I'm guessing the bug is something to do with the next few entries in the original file that bork the processing and it just dumps the rest of the input to output. (There was already a 0.0.0.0/8 at the beginning of the file, the one in the sample below would be the start of the 2nd concatenated file).

224.0.0.0/4
240.0.0.0/4
0.0.0.0/8

If you process them as individual files then it works just fine, ex: iprange -v cymru_full.netset cidr-report.netset > this_works.ipset

Hopefully it's an easy fix... lol

Let's say I have this example configuration, and I am using my own test ipset file, test1.netset.

ipv4 ipset create test1 hash:net
ipv4 ipset addfile test1 ipsets/test1.netset

ipv4 blacklist full ipset:test1

Let's say I have now added a new ip subnet to the file test1.netset.
What is the appropriate command to "reload" that ipset?

Is there a command to only reload the ipset?
Should I also optimize the ip set after changing it?

Please include a note in the documentation about the needed dependencies for packagers.

This is from iprange 1.0.5_master.

First, set up two test files:

    $ cat <<EOF >in
    1.2.3.4
    1.2.3.5
    1.2.3.6
    9.9.9.6
    9.9.9.7
    9.9.9.8
    9.9.9.9
    EOF

    $ cat <<EOF >except
    1.2.3.4
    1.2.3.5
    1.2.3.6
    EOF

Without using --reduce-factor I get the expected output:

    $ iprange in --except except >out
    $ iprange in --diff out
    1.2.3.4/31
    1.2.3.6

Using any value of --reduce-factor causes the --except to be ignored:

    $ iprange in --except except --reduce-factor 0 >out
    $ iprange in --diff out
    $ iprange in --except except --reduce-factor 10 >out
    $ iprange in --diff out
    $ iprange in --except except --reduce-factor 90 >out
    $ iprange in --diff out
    $ diff in out
    $

I'm trying to update iprange on FreeBSD ports tree from 1.0.3 to 1.0.4 and I'm getting this error on configure. Looks like the iprange.1 file is missing but I couldn't find any reference about how to create it or where to get it from.

garga@x230 ~/f/p/h/n/i/w/iprange-1.0.4 ❯❯❯ sh autogen.sh
autoreconf-2.69: Entering directory `.'
autoreconf-2.69: configure.ac: not using Gettext
autoreconf-2.69: running: aclocal --force -I m4
autoreconf-2.69: configure.ac: tracing
autoreconf-2.69: configure.ac: not using Libtool
autoreconf-2.69: running: /usr/local/bin/autoconf-2.69 --force
autoreconf-2.69: running: /usr/local/bin/autoheader-2.69 --force
autoreconf-2.69: running: automake --add-missing --copy --force-missing
configure.ac:47: installing './compile'
configure.ac:46: installing './config.guess'
configure.ac:46: installing './config.sub'
configure.ac:45: installing './install-sh'
configure.ac:45: installing './missing'
Makefile.am: installing './depcomp'
autoreconf-2.69: Leaving directory `.'
garga@x230 ~/f/p/h/n/i/w/iprange-1.0.4 ❯❯❯ ./configure
checking whether to enable maintainer-specific portions of Makefiles... no
configure: error: docs not built, use '--disable-man' or --enable-maintainer-mode

I'm using a tarball obtained from https://github.com/firehol/iprange/releases

as title. thx.

I was experimenting to see what happened if I fed iprange IPv6 addresses, somewhat predictably it failed like so:

::/8
iprange: Ignoring text after hostname '' on line 1: '::/8
'
iprange: DNS: '' failed permanently: Invalid value for hints
2001:ddd:1::/48
iprange: Ignoring text after hostname '2001' on line 2: ':ddd:1::/48
'
iprange: DNS: '2001' failed permanently: Invalid value for hints

Is there any long-term plan for some level of IPv6 support? For my personal use I'm not in any hurry but I can see how it might be useful to others. Obviously the problem space is exponentially larger, so I could also understand if the answer's "never" - useful to have in the issue list nonetheless for future searchers.