mbitsnbites / liblzg Goto Github PK
View Code? Open in Web Editor NEWliblzg is a minimal implementation of an LZ77 class compression library.
Home Page: https://gitlab.com/mbitsnbites/liblzg
License: zlib License
liblzg is a minimal implementation of an LZ77 class compression library.
Home Page: https://gitlab.com/mbitsnbites/liblzg
License: zlib License
Hi,
I want to decode the data with the LZ77. However, the buffer length is 4096 and the length of data to be decoded is 1M. 1M Data will be divided into 4096 Bytes Block, which be sent.
Each time I can read the 4096 Bytes data and decode it and flash the data. I want to know if this lib can support it or how to modify the lib?
I'd like to be able to use a custom function to allocate and free memory instead of having liblzg just call malloc/free.
AFAICT the decoder doesn't use malloc/free. For the encoder the amount of memory required is determined by the level + fast flag. I think it would be easier to have the caller pass a buffer instead of callbacks for allocating and freeing, and that would also allow a buffer to be reused across multiple operations. What do you think of something like this?
size_t LZG_WorkMemSize (lzg_encoder_config *config);
lzg_uint32_t LZG_Encode(const unsigned char *in, lzg_uint32_t insize,
unsigned char *out, lzg_uint32_t outsize,
lzg_encoder_config_t *config,
void* workmem);
Or, if you want to preserve the existing API, LZG_Encode
could call a LZG_EncodeFull
(or whatever) which would have a workmem argument.
On a related note, in _LZG_DetermineMarkers
, it seems to me that hist
is small enough that it could be placed on the stack.
after compiles this (tools/benchmark) crases on processing...
win7/64-bit using cygwin
gcc version 5.4.0 (GCC)
$ ./benchmark.exe -1 data.in
Compression: 181011 us (68 KB/s)
Floating point exception (core dumped)
$ ./benchmark.exe -memcpy data.in
Floating point exception (core dumped)
$ time ./lzg.exe -s -1 data.in data.out
real 0m0,048s / user 0m0,000s / sys 0m0,047s
$ time ./lzg.exe -1 data.in data.out
real 0m0,261s / user 0m0,062s / sys 0m0,187s
Coverity Scan flags 3 issues in liblzg with may be real. If you want, I can invite you to view the report for Squash in Coverity (detailed results are not public), where they have a handy annotated view of the source code explaining exactly how the tool arrived at the conclusion it did. Or you could create a project for liblzg. Two of the issues may have security implications (one of them is just a warning about dead code), so I'm not sure you want me to describe them here, though if you want I can.
This assert happens always with any input. Any ideas?
/* If we don't have a match at bestLength, don't even bother... */
if (UNLIKELY(pos[bestLength] == pos2[bestLength]))
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.