createrepo_c zstd compression doesn't fill in the content size, in the frame header. Python API problems. about createrepo_c HOT 4 OPEN

It kind of seems like a python-zstd issue?

sergey-dryabzhinsky/python-zstd#53 (comment)

Decompression fails where no content size is included in the frame (e.g. streaming)

...

(reply)

Yes, this module is simple and dumb. It never meant to support streaming compression. And I'll keep it this way.

createrepo_c uses streaming, so...

from createrepo_c.

Sidenote, I'll plug that it would be great to get zstd support into the Python standard library.

from createrepo_c.

If it's too hard to fix createrepo_c then flag this is super hard, backlog, or just close it.

Just kind of annoying when the only python API for that compression is very hard to use correctly.

Does it significantly impact performance if you stream to a file and then compress?

from createrepo_c.

Supposedly this works

with zstd.ZstdDecompressor().stream_reader(io.BytesIO(compressed)) as r:
    decompressed = r.read()
assert decompressed == data

indygreg/python-zstandard#150

from createrepo_c.