Code Monkey home page Code Monkey logo

lz4's Introduction

LZ4 - Extremely fast compression

LZ4 is lossless compression algorithm, providing compression speed at 400 MB/s per core, scalable with multi-cores CPU. It features an extremely fast decoder, with speed in multiple GB/s per core, typically reaching RAM speed limits on multi-core systems.

Speed can be tuned dynamically, selecting an "acceleration" factor which trades compression ratio for more speed up. On the other end, a high compression derivative, LZ4_HC, is also provided, trading CPU time for improved compression ratio. All versions feature the same decompression speed.

LZ4 library is provided as open-source software using BSD 2-Clause license.

Branch Status
master Build Status Build status coverity
dev Build Status Build status

Branch Policy:

  • The "master" branch is considered stable, at all times.
  • The "dev" branch is the one where all contributions must be merged before being promoted to master.
    • If you plan to propose a patch, please commit into the "dev" branch, or its own feature branch. Direct commit to "master" are not permitted.

Benchmarks

The benchmark uses lzbench, from @inikep compiled with GCC v7.3.0 on Linux 64-bits (Debian 4.15.17-1). The reference system uses a Core i7-6700K CPU @ 4.0GHz. Benchmark evaluates the compression of reference Silesia Corpus in single-thread mode.

Compressor Ratio Compression Decompression
memcpy 1.000 13100 MB/s 13100 MB/s
LZ4 default (v1.8.2) 2.101 730 MB/s 3900 MB/s
LZO 2.09 2.108 630 MB/s 800 MB/s
QuickLZ 1.5.0 2.238 530 MB/s 720 MB/s
Snappy 1.1.4 2.091 525 MB/s 1750 MB/s
Zstandard 1.3.4 -1 2.877 470 MB/s 1380 MB/s
LZF v3.6 2.073 380 MB/s 840 MB/s
zlib deflate 1.2.11 -1 2.730 100 MB/s 380 MB/s
LZ4 HC -9 (v1.8.2) 2.721 40 MB/s 3920 MB/s
zlib deflate 1.2.11 -6 3.099 34 MB/s 410 MB/s

LZ4 is also compatible and well optimized for x32 mode, for which it provides some additional speed performance.

Installation

make
make install     # this command may require root access

LZ4's Makefile supports standard Makefile conventions, including staged installs, redirection, or command redefinition. It is compatible with parallel builds (-j#).

Documentation

The raw LZ4 block compression format is detailed within lz4_Block_format.

To compress an arbitrarily long file or data stream, multiple blocks are required. Organizing these blocks and providing a common header format to handle their content is the purpose of the Frame format, defined into lz4_Frame_format. Interoperable versions of LZ4 must respect this frame format.

Other source versions

Beyond the C reference source, many contributors have created versions of lz4 in multiple languages (Java, C#, Python, Perl, Ruby, etc.). A list of known source ports is maintained on the LZ4 Homepage.

lz4's People

Contributors

cyan4973 avatar inikep avatar felixhandte avatar t-mat avatar terrelln avatar test4973 avatar svpv avatar kylejharper avatar bioothod avatar kou avatar lpsantil avatar ldv-alt avatar remittor avatar mgorny avatar gsauthof avatar chocobo1 avatar dcolascione avatar irwand avatar benwiederhake avatar nemequ avatar sean-purcell avatar keszybz avatar sylvestre avatar sunpoet avatar vapier avatar jwerner-chromium avatar ferdnyc avatar esiegerman avatar eberge avatar eli-b avatar

Watchers

James Cloos avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.