Comments (12)
esc
commented on July 3, 2024
I can not reproduce this error. My machine details are as follows:
- python-blosc: '1.2.1' blosc: '1.3.3'
- Linux toolbox 3.9-0.bpo.1-amd64 #1 SMP Debian 3.9.6-1~bpo70+1 x86_64 GNU/Linux
- Python 2.7.6 :: Anaconda 1.7.0 (64-bit)
@Basjo what are your machine details and how did you install python-blosc?
from python-blosc.
josephernest
commented on July 3, 2024
@esc I used
- PC Intel i5, 8 GB RAM
- Windows 7 64 bits
- Python 2.7 64 bits
blosc-1.2.1.win-amd64-py2.7.exewhich can be found here : http://www.lfd.uci.edu/~gohlke/pythonlibs/#blosc- the code pasted here + the file https://dl.dropboxusercontent.com/u/83031018/blah.bin
from python-blosc.
FrancescAlted
commented on July 3, 2024
Hmm, I am not able to reproduce this in Mac OSX, Linux or even Windows (using Python 3.3 64bit). I aways get something similar to:
C:\Users\Francesc\software\python-blosc>python prova.py
[ 0 0 2 ..., -872 -258 -599]
Here are my details:
C:\Users\Francesc\software\python-blosc>python blosc\test.py
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
python-blosc version: 1.2.2.dev
Blosc version: 1.3.3 ($Date:: 2014-02-15 #$)
Compressors available: ['blosclz', 'lz4', 'lz4hc']
Compressor library versions:
BloscLZ: 1.0.1
LZ4: 1.1.3
Python version: 3.3.1 (v3.3.1:d9893d13c628, Apr 6 2013, 20:30:21) [MSC v.1600
4 bit (AMD64)]
Platform: Windows-7-AMD64 (6.1.7601)
Processor: Intel64 Family 6 Model 58 Stepping 9, GenuineIntel
Byte-ordering: little
Detected cores: 4
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
test_all_compressors (__main__.TestCodec) ... ok
test_basic_codec (__main__.TestCodec) ... ok
test_compress_exceptions (__main__.TestCodec) ... ok
test_compress_ptr_exceptions (__main__.TestCodec) ... ok
test_decompress_exceptions (__main__.TestCodec) ... ok
test_decompress_ptr_exceptions (__main__.TestCodec) ... ok
test_get_clib (__main__.TestCodec) ... ok
test_pack_array_exceptions (__main__.TestCodec) ... ok
test_set_nthreads_exceptions (__main__.TestCodec) ... ok
test_unpack_array_exceptions (__main__.TestCodec) ... ok
compress (blosc.toplevel)
Doctest: blosc.toplevel.compress ... ok
compress_ptr (blosc.toplevel)
Doctest: blosc.toplevel.compress_ptr ... ok
decompress (blosc.toplevel)
Doctest: blosc.toplevel.decompress ... ok
decompress_ptr (blosc.toplevel)
Doctest: blosc.toplevel.decompress_ptr ... ok
free_resources (blosc.toplevel)
Doctest: blosc.toplevel.free_resources ... ok
pack_array (blosc.toplevel)
Doctest: blosc.toplevel.pack_array ... ok
set_nthreads (blosc.toplevel)
Doctest: blosc.toplevel.set_nthreads ... ok
unpack_array (blosc.toplevel)
Doctest: blosc.toplevel.unpack_array ... ok
----------------------------------------------------------------------
Ran 18 tests in 0.504s
OK
I would say there should be something wrong in the 2.7 build for Win.
from python-blosc.
esc
commented on July 3, 2024
@FrancescAlted can you test the the pre-built binaries? I tried accessing the page, but it seems down right now.
from python-blosc.
FrancescAlted
commented on July 3, 2024
@esc It is not exactly down, but very slow. @Basjo could you try with Python 3.3 in your computer?
from python-blosc.
FrancescAlted
commented on July 3, 2024
Checked with Python 2.7 and MinGW (installed via Anaconda):
C:\Users\Francesc\software\python-blosc>python prova.py
[ 0 0 2 ..., -872 -258 -599]
C:\Users\Francesc\software\python-blosc>python blosc\test.py
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
python-blosc version: 1.2.2.dev
Blosc version: 1.3.3 ($Date:: 2014-02-15 #$)
Compressors available: ['blosclz', 'lz4', 'lz4hc']
Compressor library versions:
BloscLZ: 1.0.1
LZ4: 1.1.3
Python version: 2.7.4 |Anaconda 1.8.0 (32-bit)| (default, Apr 9 2013, 12:19:24)
[MSC v.1500 32 bit (Intel)]
Platform: Windows-7-AMD64 (6.1.7601)
Processor: Intel64 Family 6 Model 58 Stepping 9, GenuineIntel
Byte-ordering: little
Detected cores: 4
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
test_all_compressors (__main__.TestCodec) ... ok
test_basic_codec (__main__.TestCodec) ... ok
test_compress_exceptions (__main__.TestCodec) ... ok
test_compress_ptr_exceptions (__main__.TestCodec) ... ok
test_decompress_exceptions (__main__.TestCodec) ... ok
test_decompress_ptr_exceptions (__main__.TestCodec) ... ok
test_get_clib (__main__.TestCodec) ... ok
test_pack_array_exceptions (__main__.TestCodec) ... ok
test_set_nthreads_exceptions (__main__.TestCodec) ... ok
test_unpack_array_exceptions (__main__.TestCodec) ... ok
compress (blosc.toplevel)
Doctest: blosc.toplevel.compress ... ok
compress_ptr (blosc.toplevel)
Doctest: blosc.toplevel.compress_ptr ... ok
decompress (blosc.toplevel)
Doctest: blosc.toplevel.decompress ... ok
decompress_ptr (blosc.toplevel)
Doctest: blosc.toplevel.decompress_ptr ... ok
free_resources (blosc.toplevel)
Doctest: blosc.toplevel.free_resources ... ok
pack_array (blosc.toplevel)
Doctest: blosc.toplevel.pack_array ... ok
set_nthreads (blosc.toplevel)
Doctest: blosc.toplevel.set_nthreads ... ok
unpack_array (blosc.toplevel)
Doctest: blosc.toplevel.unpack_array ... ok
----------------------------------------------------------------------
Ran 18 tests in 0.411s
OK
Everything seems fine here too.
from python-blosc.
josephernest
commented on July 3, 2024
Hi @FrancescAlted, could you send me your latest installer for that ? Then I could try with your latest version, maybe it would solve it.
Important info : I tried it with Python 2.7 32bits, and it works : no error.
The error only comes when using Python 2.7 64bits.
However the test.py works fine even with Python 2.7 64bits :
test_all_compressors (__main__.TestCodec) ... ok
test_basic_codec (__main__.TestCodec) ... ok
test_compress_exceptions (__main__.TestCodec) ... ok
test_compress_ptr_exceptions (__main__.TestCodec) ... ok
test_decompress_exceptions (__main__.TestCodec) ... ok
test_decompress_ptr_exceptions (__main__.TestCodec) ... ok
test_get_clib (__main__.TestCodec) ... ok
test_pack_array_exceptions (__main__.TestCodec) ... ok
test_set_nthreads_exceptions (__main__.TestCodec) ... ok
test_unpack_array_exceptions (__main__.TestCodec) ... ok
compress (blosc.toplevel)
Doctest: blosc.toplevel.compress ... ok
compress_ptr (blosc.toplevel)
Doctest: blosc.toplevel.compress_ptr ... ok
decompress (blosc.toplevel)
Doctest: blosc.toplevel.decompress ... ok
decompress_ptr (blosc.toplevel)
Doctest: blosc.toplevel.decompress_ptr ... ok
free_resources (blosc.toplevel)
Doctest: blosc.toplevel.free_resources ... ok
pack_array (blosc.toplevel)
Doctest: blosc.toplevel.pack_array ... ok
set_nthreads (blosc.toplevel)
Doctest: blosc.toplevel.set_nthreads ... ok
unpack_array (blosc.toplevel)
Doctest: blosc.toplevel.unpack_array ... -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
python-blosc version: 1.2.1
Blosc version: 1.3.3 ($Date:: 2014-02-15 #$)
Compressors available: ['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib']
Compressor library versions:
BloscLZ: 1.0.1
LZ4: 1.1.3
Snappy: 1.1.1
Zlib: 1.2.8
Python version: 2.7.6 (default, Nov 10 2013, 19:24:24) [MSC v.1500 64 bit (AMD64)]
Platform: Windows-7-AMD64 (6.1.7600)
Processor: Intel64 Family 6 Model 42 Stepping 7, GenuineIntel
Byte-ordering: little
Detected cores: 4
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
ok
----------------------------------------------------------------------
Ran 18 tests in 0.249s
OK
from python-blosc.
FrancescAlted
commented on July 3, 2024
Hi @Basjo, I tried the binaries at http://www.lfd.uci.edu/~gohlke/pythonlibs/#blosc, and I can reproduce your error (however, using other compressors instide Blosc like blosclz, snappy and zlib work perfectly fine).
Sorry, but I cannot compile Blosc for Python 2.7 because I only have Visual Studio 2010/2012, and for this I would need 2008. But for Python 3.3 64-bit and VS2010 everything is fine here, so I think the best would be to report this ticket to Christoph Gohlke.
from python-blosc.
josephernest
commented on July 3, 2024
Hi @FrancescAlted Yes it's the same for me : the error is only for lz4. When using blosclz, snappy, or zlib, no error.
I could report this ticket to C. Gohlke, but I don't think it would change anything : the binaries were properly compiled (when I run test.py, all is "OK"), so if he compiles again the same code, it will give the same result, don't you think so ?
What is the internal blosc reason for a blosc_extension.error: Error -1 while compressing data, what could it be ?
By the way, when you try with shuffle = False, do you get an error as well ?
from python-blosc.
esc
commented on July 3, 2024
-1 is the error that blosc returns in case anything went wrong. This can happen in a multiple of error conditions, not enough space, buffer overrun and others.
You need to grep for -1 in:
https://github.com/Blosc/c-blosc/blob/master/blosc/blosc.c
from python-blosc.
FrancescAlted
commented on July 3, 2024
Yes, I discovered what I think was the root of the problem (incidentally, this affected to BLZ too: ContinuumIO/blz#5). The fix was basically this one:
Closing this. @Basjo in case you are still getting the error on Win, please reopen this ticket.
I have released new versions of c-blosc (1.3.4) and python-blosc (1.2.2) with this fix in.
from python-blosc.
josephernest
commented on July 3, 2024
Thanks @FrancescAlted, problem solved !
from python-blosc.
Related Issues (20)
- Issues decompressing bytes from files HOT 1
- Replace obsolete `popen2` HOT 1
- Properly identify vendored `cpuinfo.py` version
- Blosc_ROOT cmake warning: Policy CMP0074 is not set HOT 2
- "RuntimeError: Cannot decompress" for a compressed sequence of more than 7240 zero bytes HOT 1
- Very bad compression on short inputs 1-127 bytes long HOT 5
- “python_requires” should be set with “>=3.6”, as blosc 1.10.6 is not compatible with all Python versions. HOT 2
- wrong setuptools build command
- Concatenate two blosc compressed bytes objects HOT 2
- LICENSES/BLOSC.txt HOT 4
- Rename default branch HOT 1
- Update pypi with latest blosc version HOT 3
- Wheel for Python 3.10 and Python 3.11 HOT 3
- Cannot install blosc 1.11.0 on apple M1 machine HOT 3
- decompress in fore-end HOT 1
- README link to python-blosc2 seems useful HOT 1
- __pack_tensor__ must be made portable and not depend on Python HOT 2
- __pack_tensor__ should be in the beginning of the file to avoid seeking the whole file HOT 2
- Python 3.12 compatibility HOT 6
- Numpy 2 compatibility
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from python-blosc.