tmaklin / bxzstr Goto Github PK

I'm currently still using gzstream, but was looking for an updated interface and bxzstr looks pretty good!
However, the config.hpp file could potentially cause ambiguity depending on what the final include path looks like.
Would it be possible to rename as bxzstr_config.hpp (for example), to preclude any such problems (or did I miss something)?

It would be great to have a replacement function to calculate the uncompressed size. Usually people do something like that:

  bxz::ifstream is(filename, std::ios::binary);
  is.seekg (0, std::ios::end);
  length = UInt(is.tellg());
  is.seekg (0, std::ios::beg);

However with bxzstr seeking from end is not supported. However the decompressed size can be retrieved with ZSTD_getFrameContentSize for example (when using ZStd).
Are there any plans to add such a function

When run with the code

#include "bxzstr.hpp"
#include <iostream>

int main(int argc, char** argv){
    bxz::ofstream out("test.gz", bxz::z, 1);
    for(long long i = 0; i < 10; i++){
        out << i << "\n";
    }
    out.close();
}

closing the file will not write anything (test.gz will be zero bytes). Flushing the stream and then closing will perform the write but using either separately fails.

Hi 👋

When I try to compile this with exceptions turned off -fno-exceptions then it errors.

Is this an explicit decision, or could there be an option for having no exception support?

Would that make sense to have a new release?

The function ZSTD_defaultCLevel was not part of any release prior to v1.5.0 of Zstd, therefore:

/home/dmg/.cpm/bxzstr/d488119fdf94185eb9eaac489709a9e37dbe9c76/include/zstd_stream_wrapper.hpp:41:22: error: use of undeclared identifier 'ZSTD_defaultCLevel'; did you mean 'ZSTD_maxCLevel'?
                        const int level = ZSTD_defaultCLevel(), const int = 0)

Since Ubuntu 22.04 comes with ZStd 1.4.8 this seems like a reasonable thing to try and fix in the meantime

See if libdeflate can be used in place of zlib when configured to do so. Libdeflate provides a more optimized implementation for DEFLATE than zlib (for some cases) but the API is not 1-to-1 compatible, so adding support might require some work.

when create bxz::ofstream, parameters as ("/tmp/test2.txt", std::ios_base::out, bxz::plaintext, 0),

" what(): Unrecognized compression type. " will be thrown.

See #14 for a usage example.

zstd (https://facebook.github.io/zstd/) provides excellent compression speed & ratios and an API similar to zlib, which should make it relatively easy to implement.

Adding a new algorithm requires creating a new header-file which inherits the stream_wrapper class and implements the functions from stream_wrapper.hpp. This is not documented anywhere and should be added so adding support for new algorithms is easier.

The code should be tested extensively and automatically before the first "stable" release is made released.

Would be nice to put strict_stream namespace inside the bxz namespace.

Sorry for not providing a pull request as it is trivial.

It would be nice to state at least approximately the minimal versions of the supported libraries.
E.g. zstd>1.4.0

Hi @tmaklin ,

I run some simple codes under valgrind and see memory leaks originated in the bz_stream_wrapper.hpp and z_stream_wrapper.hpp. I have submitted an obvious fix #5 , but it seems it is not enough and one should actually have bz_stream::next_in = NULL; in the initialization. for all the cases.
But without a deeper knowledge of the code it is hard to judge.
Would you have a look?

Also, the z_stream_wrapper.hpp has the same issues.

Best regards,

Andrii