rygorous / ryg_rans Goto Github PK
View Code? Open in Web Editor NEWSimple rANS encoder/decoder (arithmetic coding-ish entropy coder).
License: Other
Simple rANS encoder/decoder (arithmetic coding-ish entropy coder).
License: Other
Hi, the sse4.1 version is awesome. Is it possition to provide a neon version for mobile platform?
I want to use rans64.h in arm android device, but got error like this:
rans64.h:44:35: error: __int128 is not supported on this target.
how can I fix the bug?
Lines 80 to 81 in 72d6315
Does this mean to say "from the last byte backward to the first byte" instead of "from beginning to end"?
I stumbled upon this error trying to use different test files (#9). It turned out, it is caused by a size of input. Mozilla from Silesia corpus is enough to cause this error even though it uses ~133MB of RAM with the file.
$ g++ -o rygrans-exam main.cpp -g -O3 -Wno-literal-suffix -lm -lrt
$ ln -fs mozilla book1
$ gdb --args rygrans-exam mozilla
(gdb) r
rANS encode:
1184805127 clocks, 23.1 clocks/symbol ( 175.4MiB/s)
(...)
rANS: 39840007 bytes
1061714280 clocks, 20.7 clocks/symbol ( 184.1MiB/s)
(...)
ERROR: bad decoder!
interleaved rANS encode:
927790930 clocks, 18.1 clocks/symbol ( 196.3MiB/s)
(...)
interleaved rANS: 39840009 bytes
1018967059 clocks, 19.9 clocks/symbol ( 187.7MB/s)
(...)
ERROR: bad decoder!
*** glibc detected *** ~/ryg_rans/rygrans-exam: free(): invalid pointer: 0xb2b7f008 ***
======= Backtrace: =========
/lib/libc.so.6(+0x78027)[0xb7cef027]
/usr/lib/libstdc++.so.6(_ZdlPv+0x18)[0xb7eb5878]
/usr/lib/libstdc++.so.6(_ZdaPv+0x18)[0xb7eb58d8]
~/ryg_rans/rygrans-exam[0x804960b]
/lib/libc.so.6(__libc_start_main+0xf5)[0xb7c905a5]
~/ryg_rans/rygrans-exam[0x8049805]
======= Memory map: ========
08048000-0804b000 r-xp 00000000 07:4b 63951 ~/ryg_rans/rygrans-exam
0804b000-0804c000 rw-p 00002000 07:4b 63951 ~/ryg_rans/rygrans-exam
0804c000-08071000 rw-p 00000000 00:00 0 [heap]
afaa5000-b7c5d000 rw-p 00000000 00:00 0
(...)
Program received signal SIGABRT, Aborted.
0xb7ca6af5 in raise () from /lib/libc.so.6
(gdb) bt
#0 0xb7ca6af5 in raise () from /lib/libc.so.6
#1 0xb7ca8453 in abort () from /lib/libc.so.6
#2 0xb7ce74b5 in ?? () from /lib/libc.so.6
#3 0xb7cef027 in ?? () from /lib/libc.so.6
#4 0xb7eb5878 in operator delete(void*) () from /usr/lib/libstdc++.so.6
#5 0xb7eb58d8 in operator delete[](void*) () from /usr/lib/libstdc++.so.6
#6 0x0804960b in main () at main.cpp:293
(gdb) l main.cpp:293
288 if (memcmp(in_bytes, dec_bytes, in_size) == 0)
289 printf("decode ok!\n");
290 else
291 printf("ERROR: bad decoder!\n");
292
293 delete[] out_buf;
294 delete[] dec_bytes;
295 delete[] in_bytes;
296 return 0;
297 }
If the input is composed of only one character, e.g. "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", the compressed output size is double the input size:
ANS input size: 72, output size 148, compression ratio -105.555556%
If another character is introduced, e.g. "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaab", the excellent compression ratio is restored:
ANS input size: 72, output size 4, compression ratio 94.444444%
I tested it on the scalar version of rans_word_sse41.h
.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.