It is possible cache the bloom filter to NVMe? (support filters larger than 2^38 bytes) about libfilter HOT 4 OPEN

You could use mmap in the C interface to allocate a libfilter_region's block pointer, I suppose. If you just accessed the NVMe space needed on each read or write, I suppose the OS might cache it anyway.

Thoughts?

from libfilter.

You could use mmap in the C interface to allocate a libfilter_region's block pointer, I suppose. If you just accessed the NVMe space needed on each read or write, I suppose the OS might cache it anyway.

Thoughts?

I will have a try.
Thank you!

from libfilter.

Looking more carefully at the code, I don't think it supports filters larger than 2^32 bytes. I'll take this as a feature request.

from libfilter.

When I run the following, the correct amount of memory is allocated on my machine:

BlockFilter::CreateWithNdvFpp(1ul << 34, 0.0001)

How much memory is allocated on your machine if you just call this factory method and then return from main? What tool are you using to measure the amount of memory allocated?

from libfilter.