An efficient and light-weight ordered set of 32 bits integers. This is a Python wrapper for the C library CRoaring.

You can use a bitmap nearly as the classical Python set in your code:

from pyroaring import BitMap
bm1 = BitMap()
bm1.add(3)
bm1.add(18)
bm2 = BitMap([3, 27, 42])
print("bm1       = %s" % bm1)
print("bm2       = %s" % bm2)
print("bm1 & bm2 = %s" % (bm1&bm2))
print("bm1 | bm2 = %s" % (bm1|bm2))

Output:

bm1       = BitMap([3, 18])
bm2       = BitMap([3, 27, 42])
bm1 & bm2 = BitMap([3])
bm1 | bm2 = BitMap([3, 18, 27, 42])

Note: this installation method requires a recent C compiler like GCC.

Supported systems:

• Linux and MacOS: Python 2.7 or Python 3.4 or higher.
• Windows: Python 3.4 or higher.

To install pyroaring on your local account, use the following command:

pip install pyroaring --user

For a system-wide installation, use the following command:

pip install pyroaring

Naturally, the latter may require superuser rights (consider prefixing the commands by sudo).

If you want to use Python 3 and your system defaults on Python 2.7, you may need to adjust the above commands, e.g., replace pip by pip3.

Several wheels are published on GitHub for each release: https://github.com/Ezibenroc/PyRoaringBitMap/releases

Installing from a wheel should be the easiest as no C compiler is required. However, performance may be lower. Note that you have to chose the right wheel, depending on your system.

For instance, to install pyroaring version 0.2.1 for Python 3.6 on Linux:

pip install --user https://github.com/Ezibenroc/PyRoaringBitMap/releases/download/0.2.1/pyroaring-0.2.1-cp36-cp36m-linux_x86_64.whl

If you want to compile (and install) pyroaring by yourself, for instance to modify the Cython sources or because you do not have pip, follow these steps.

Note that the Python package Cython is required. You may install it as:

pip install --upgrade setuptools -user
pip install cython --user

Clone this repository.

git clone https://github.com/Ezibenroc/PyRoaringBitMap.git
cd PyRoaringBitMap
git submodule init && git submodule update

Build pyroaring locally, e.g. to test a new feature you made.

python setup.py build_ext -i

On macOS this may fail with errors because setuptools adds -arch x86_64 -arch i386 to the compiler command, which may conflict with the -march=native flag. You can overwrite this behavior by setting the ARCHFLAGS flag:

ARCHFLAGS="" python setup.py build_ext -i

Then you can test the new code:

pip install hypothesis --user
python test.py # run the tests, optional but recommended

Install pyroaring (use this if you do not have pip).

python setup.py install # may require superuser rights, add option --user if you wish to install it on your local account

Package pyroaring.

python setup.py sdist
pip install dist/pyroaring-0.1.?.tar.gz # optionnal, to install the package

Build a wheel.

python setup.py bdist_wheel

For all the above commands, two environment variables can be used to control the compilation.

• DEBUG=1 to build pyroaring in debug mode.
• ARCHI=<cpu-type> to build pyroaring for the given platform. The platform may be any keyword given to the -march option of gcc (see the documentation). Note that cross-compiling for a 32-bit architecture from a 64-bit architecture is not supported.

Example of use:

DEBUG=1 ARCHI=x86-64 python setup.py build_ext

Pyroaring is compared with the built-in set and other implementations:

• A Python wrapper of CRoaring called python-croaring
• A Cython implementation of Roaring bitmaps called roaringbitmap
• A Python implementation of ordered sets called sortedcontainers

The script quick_bench.py measures the time of different set operations. It uses randomly generated sets of size 1e6 and density 0.125. For each operation, the average time (in seconds) of 30 tests is reported.

The results have been obtained with:

• CPU Intel Xeon CPU E5-2630 v3
• CPython version 3.5.3
• gcc version 6.3.0
• Cython version 0.28.3
• pyroaring commit dcf448a
• python-croaring commit 3aa61dd
• roaringbitmap commit 502d78d
• sortedcontainers commit 7d6a28c