mbitsnbites / liblzg Goto Github PK

Hi,
I want to decode the data with the LZ77. However, the buffer length is 4096 and the length of data to be decoded is 1M. 1M Data will be divided into 4096 Bytes Block, which be sent.
Each time I can read the 4096 Bytes data and decode it and flash the data. I want to know if this lib can support it or how to modify the lib?

I'd like to be able to use a custom function to allocate and free memory instead of having liblzg just call malloc/free.

AFAICT the decoder doesn't use malloc/free. For the encoder the amount of memory required is determined by the level + fast flag. I think it would be easier to have the caller pass a buffer instead of callbacks for allocating and freeing, and that would also allow a buffer to be reused across multiple operations. What do you think of something like this?

size_t LZG_WorkMemSize (lzg_encoder_config *config);

lzg_uint32_t LZG_Encode(const unsigned char *in, lzg_uint32_t insize,
                        unsigned char *out, lzg_uint32_t outsize,
                        lzg_encoder_config_t *config,
                        void* workmem);

Or, if you want to preserve the existing API, LZG_Encode could call a LZG_EncodeFull (or whatever) which would have a workmem argument.

On a related note, in _LZG_DetermineMarkers, it seems to me that hist is small enough that it could be placed on the stack.

after compiles this (tools/benchmark) crases on processing...

win7/64-bit using cygwin
gcc version 5.4.0 (GCC)

$ ./benchmark.exe -1 data.in
Compression: 181011 us (68 KB/s)
Floating point exception (core dumped)

$ ./benchmark.exe -memcpy data.in
Floating point exception (core dumped)

$ time ./lzg.exe -s -1 data.in data.out
real 0m0,048s / user 0m0,000s / sys 0m0,047s

$ time ./lzg.exe -1 data.in data.out
real 0m0,261s / user 0m0,062s / sys 0m0,187s

Coverity Scan flags 3 issues in liblzg with may be real. If you want, I can invite you to view the report for Squash in Coverity (detailed results are not public), where they have a handy annotated view of the source code explaining exactly how the tool arrived at the conclusion it did. Or you could create a project for liblzg. Two of the issues may have security implications (one of them is just a warning about dead code), so I'm not sure you want me to describe them here, though if you want I can.

This assert happens always with any input. Any ideas?

/* If we don't have a match at bestLength, don't even bother... */
if (UNLIKELY(pos[bestLength] == pos2[bestLength]))