Possible Breaking changes to make about croaring-rs HOT 7 CLOSED

Maybe From::from rather than Bitmap::of? (so Bitmap::from(&[0, 1, 2]) rather than of) (and add From<[u32; N]>)

from croaring-rs.

Should we also try to future proof ourselves to have a ViewDeserializer vs UncheckedViewDeserializer (I'm expecting a safe view function at some point)

I've looked into providing a safe view deserialization function in CRoaring. My intention was to add two checks:

1. Guard against buffer overflows. This is fairly easy, though a bit cumbersome.
2. Validate alignment. My assumption was that if the input buffer was aligned, then all accesses would be aligned. Unfortunately this turned out not to be true. Because the runFlagBitset in the header has a variable number of bytes, this can cause unaligned accesses, even if the input buffer is aligned.

Unfortunately I don't see a good way of solving (2) without changing the spec, e.g. padding runFlagBitset to ensure alignment. As such, I don't think we should hold our breath on a safe view function (though I'd love to see it!) 😢

Generally speaking, I'd love to see a 1.0 release. My motivation is to use croaring via plrust in AWS RDS Postgres. I believe the next RDS plrust release is 1-3 months away, and I'd love to see croaring v1.0 supported rather than v0.9. That being said, I acknowledge that breaking changes should not be rushed. If I can help, let me know 😊

from croaring-rs.

Thinking we might want to add the ability to serialize treemaps into any writer so we don't necessarily need to keep it all in memory. We'll probably want both into any writer, and into a Vec, since for Frozen we need to ensure alignment, and we can't guarantee the writer will start aligned.

from croaring-rs.

I don't see a deserialize from reader in this implementation. roaring-rs has one (although it's very slow).

from croaring-rs.

Correct, there is not one in CRoaring. But we can write each bitmap in a treemap without keeping the full serialized treemap in memory, and the caller could use mmap to read the treemap without requiring as much heap memory use. Not super useful, but might as well since we can support it, and if CRoaring ever does get streaming ser/deserialization, we can use that for more wins.

from croaring-rs.

Is there a view from bytes for treemaps?

from croaring-rs.

No, sorry, I'm still not being clear, just to avoid two copies on the heap, you could deserialize from a file backed mmap, but there's not currently a way of creating a view for treemaps over an existing byte slice.

from croaring-rs.