Code Monkey home page Code Monkey logo

ryg_rans's People

Contributors

rygorous avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ryg_rans's Issues

Neon Support

Hi, the sse4.1 version is awesome. Is it possition to provide a neon version for mobile platform?

Typo in the comment (?)

ryg_rans/rans_byte.h

Lines 80 to 81 in 72d6315

// NOTE: With rANS, you need to encode symbols in *reverse order*, i.e. from
// beginning to end! Likewise, the output bytestream is written *backwards*:

Does this mean to say "from the last byte backward to the first byte" instead of "from beginning to end"?

error when decompressing big files

I stumbled upon this error trying to use different test files (#9). It turned out, it is caused by a size of input. Mozilla from Silesia corpus is enough to cause this error even though it uses ~133MB of RAM with the file.

$ g++ -o rygrans-exam main.cpp -g -O3 -Wno-literal-suffix -lm -lrt
$ ln -fs mozilla book1
$ gdb --args rygrans-exam mozilla

(gdb) r

rANS encode:
1184805127 clocks, 23.1 clocks/symbol ( 175.4MiB/s)
(...)
rANS: 39840007 bytes
1061714280 clocks, 20.7 clocks/symbol ( 184.1MiB/s)
(...)
ERROR: bad decoder!

interleaved rANS encode:
927790930 clocks, 18.1 clocks/symbol ( 196.3MiB/s)
(...)
interleaved rANS: 39840009 bytes
1018967059 clocks, 19.9 clocks/symbol ( 187.7MB/s)
(...)
ERROR: bad decoder!
*** glibc detected *** ~/ryg_rans/rygrans-exam: free(): invalid pointer: 0xb2b7f008 ***
======= Backtrace: =========
/lib/libc.so.6(+0x78027)[0xb7cef027]
/usr/lib/libstdc++.so.6(_ZdlPv+0x18)[0xb7eb5878]
/usr/lib/libstdc++.so.6(_ZdaPv+0x18)[0xb7eb58d8]
~/ryg_rans/rygrans-exam[0x804960b]
/lib/libc.so.6(__libc_start_main+0xf5)[0xb7c905a5]
~/ryg_rans/rygrans-exam[0x8049805]
======= Memory map: ========
08048000-0804b000 r-xp 00000000 07:4b 63951      ~/ryg_rans/rygrans-exam
0804b000-0804c000 rw-p 00002000 07:4b 63951      ~/ryg_rans/rygrans-exam
0804c000-08071000 rw-p 00000000 00:00 0          [heap]
afaa5000-b7c5d000 rw-p 00000000 00:00 0 
(...)

Program received signal SIGABRT, Aborted.
0xb7ca6af5 in raise () from /lib/libc.so.6
(gdb) bt
#0  0xb7ca6af5 in raise () from /lib/libc.so.6
#1  0xb7ca8453 in abort () from /lib/libc.so.6
#2  0xb7ce74b5 in ?? () from /lib/libc.so.6
#3  0xb7cef027 in ?? () from /lib/libc.so.6
#4  0xb7eb5878 in operator delete(void*) () from /usr/lib/libstdc++.so.6
#5  0xb7eb58d8 in operator delete[](void*) () from /usr/lib/libstdc++.so.6
#6  0x0804960b in main () at main.cpp:293
(gdb) l main.cpp:293
288         if (memcmp(in_bytes, dec_bytes, in_size) == 0)
289             printf("decode ok!\n");
290         else
291             printf("ERROR: bad decoder!\n");
292
293         delete[] out_buf;
294         delete[] dec_bytes;
295         delete[] in_bytes;
296         return 0;
297     }

Single symbol explosion

If the input is composed of only one character, e.g. "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", the compressed output size is double the input size:

ANS input size: 72, output size 148, compression ratio -105.555556%

If another character is introduced, e.g. "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaab", the excellent compression ratio is restored:
ANS input size: 72, output size 4, compression ratio 94.444444%

I tested it on the scalar version of rans_word_sse41.h.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.