Comments (60)
jcfr
commented on July 3, 2024
2
Step 1 is the simplification of the build system so that we can leverage the fact c-blosc is already using CMake. This is a work in progress but it gives a good idea.
See master...jcfr:simplify-buildsystem
Next steps:
- support building against an existing install of blosc
- test on windows and macos
- add support for automatic building and upload of wheel
Cc: @thewtex
from python-blosc.
matthew-brett
commented on July 3, 2024
1
Hi Francesc - welcome back - I've added you ...
from python-blosc.
esc
commented on July 3, 2024
1
Awesome, thanks for your message! Happy to see such interest in the package, really glad you like it!
This has been on my list for a long time and I'm still very much interested in building these packages. We have constructed quite some tooling but nothing has yet successfully yielded the binary wheels. If you can build them somehow and give me access I will happily upload them to PyPi.
Also, there have been a number of issues with the underlying Blosc library in the last few months, but with the most recent release it should be all good. You could also look at the existing tooling and see if you can debug the Travis builds. Let us know if you need any pointers!
…On 16 January 2019 05:19:54 CET, Matt McCormick ***@***.***> wrote:
Hi there,
I discovered this package and was very impressed! I would like to use
it as a dependency, but the lack of binary packages prevents this. If
we added [scikit-build](https://scikit-build.readthedocs.io/en/latest/)
configuration to help build the wheels and
[scikit-ci](https://scikit-ci.readthedocs.io/en/latest/) configuration
to build per platform, would they be integrated and the packages pushed
to PyPI?
CC: @jcfr
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#131 (comment)
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
from python-blosc.
matthew-brett
commented on July 3, 2024
1
Yes, I was going to suggest scikit-build if you're using CMake, but I see you're already trying that ...
from python-blosc.
thewtex
commented on July 3, 2024
1
#204 adds initial wheel support.
from python-blosc.
robbmcleod
commented on July 3, 2024
@matthew-brett do you think it would be reasonable to add python-blosc to the manywheels ecosystem? I don't know what sort of restrictions you have on your content provider but I could certainly maintain the associated MacPython repo.
from python-blosc.
matthew-brett
commented on July 3, 2024
@robbmcleod - yes - that's a good idea - no problem with resources I don't think. Do you want to use MacPython or this org for the repository?
from python-blosc.
esc
commented on July 3, 2024
That would be awesome!
…On 5 October 2017 19:08:48 CEST, Matthew Brett ***@***.***> wrote:
@robbmcleod - yes - that's a good idea - no problem with resources I
don't think. Do you want to use MacPython or this org for the
repository?
--
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub:
#131 (comment)
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
from python-blosc.
robbmcleod
commented on July 3, 2024
@matthew-brett: I would say here but that's up to Valentin and Francesc. I talked about adding manywheels support for blosc with Francesc last week so I know he's on board.
Another question would be support for the AVX2 instruction set, if the wheels are compiled against CentOS 6.0 and GCC 4.X? I don't know if there's any way to make pip try and install from source first and if that fails use the wheel? That would be helpful for numexpr too.
@esc do you have admin rights here, or would you prefer MacPython to host? If here we would need a python-blosc-wheels repo with rights.
from python-blosc.
esc
commented on July 3, 2024
https://github.com/Blosc/python-blosc-wheels <-- no problem!
from python-blosc.
robbmcleod
commented on July 3, 2024
I made the initial commit Blosc/python-blosc-wheels@811d5b6. @matthew-brett we'll need a RSA key for .travis.yml and appveyor.yml. And someone with access to the Blosc CI accounts needs to setup those hooks and see if it works.
from python-blosc.
esc
commented on July 3, 2024
I have access to Travis-CI do you have any instructions for what needs to be configured?
from python-blosc.
matthew-brett
commented on July 3, 2024
Here's the Rackspace key for travis:
For Appveyor - can you make a team, and add me to it?
https://www.appveyor.com/docs/team-setup/
Then I can log in with that team and make the encrypted Appveyor line as well.
from python-blosc.
esc
commented on July 3, 2024
I don't know anything about Appveyor. I think we will need help from @FrancescAlted
from python-blosc.
matthew-brett
commented on July 3, 2024
I can do it later if you like. It's not too hard, just requires jumping through more hoops than you would guess before you start.
from python-blosc.
esc
commented on July 3, 2024
O.K. please do it. Also, do i need to activate the travis build for the wheels repo?
from python-blosc.
matthew-brett
commented on July 3, 2024
OK - I've made an account. @esc @robbmcleod - would you mind sending me your email addresses by email [email protected] so I can add you to the Appveyor team?
@esc - yes please do enable travis builds on the repo. I like to use the travis command line client and travis enable but of course the web interface is fine too.
from python-blosc.
esc
commented on July 3, 2024
Done and done.
…On 7 October 2017 16:07:38 CEST, Matthew Brett ***@***.***> wrote:
OK - I've made an account. @esc @robbmcleod - would you mind sending
me your email addresses by email ***@***.*** so I can add
you to the Appveyor team?
@esc - yes please do enable travis builds on the repo. I like to use
the travis command line client and `travis enable` but of course the
web interface is fine too.
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#131 (comment)
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
from python-blosc.
matthew-brett
commented on July 3, 2024
@esc - I can't find an email from you - would you mind resending. @robbmcleod - I've added you as an administrator ...
from python-blosc.
esc
commented on July 3, 2024
Sent now.
…On 11 October 2017 10:52:38 CEST, Matthew Brett ***@***.***> wrote:
@esc - I can't find an email from you - would you mind resending.
@robbmcleod - I've added you as an administrator ...
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#131 (comment)
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
from python-blosc.
matthew-brett
commented on July 3, 2024
Ah - it went straight to my gmail spam folder because it violated some guidelines of theirs. Sorry, I didn't save the full message - but just to let you know. I have added you now. I think you should be able to do anything with that account.
from python-blosc.
FrancescAlted
commented on July 3, 2024
Hi. Coming back from some holidays; is good to see this effort going on. @matthew-brett could you please add me to the AppVeyor team? ([email protected]).
from python-blosc.
esc
commented on July 3, 2024
Any more input from my end needed?
from python-blosc.
esc
commented on July 3, 2024
Kindly asking again: anything else needed for this from our end? I'd be happy to do whatever is needed to move this forward.
from python-blosc.
matthew-brett
commented on July 3, 2024
Sorry - I'm lost - @esc - what / who are you waiting for?
from python-blosc.
esc
commented on July 3, 2024
The binaries to arrive on PyPi.
…On 26 October 2017 11:30:25 CEST, Matthew Brett ***@***.***> wrote:
Sorry - I'm lost - @esc - what / who are you waiting for?
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#131 (comment)
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
from python-blosc.
matthew-brett
commented on July 3, 2024
So - the steps are
- make python-blosc-wheels repo upload to Rackspace
- download from Rackspace / upload to PyPI (usually with wheelhouse-uploader command).
I can't see the wheels at https://3f23b170c54c2533c070-1c8a9b3114517dc5fe17b7c3f8c63a43.ssl.cf2.rackcdn.com so I guess the first step isn't working yet?
The second step only needs someone with PyPI upload permissions.
from python-blosc.
robbmcleod
commented on July 3, 2024
The first step is to turn on TravisCI and Appeveyor for the https://github.com/Blosc/python-blosc-wheels repo.
from python-blosc.
esc
commented on July 3, 2024
There were some issues with the python-blosc-wheels repo, I started fixing a few things and now at least some of the builds go green.
from python-blosc.
esc
commented on July 3, 2024
The remaining red jobs here:
https://travis-ci.org/Blosc/python-blosc-wheels/builds/294174424
mostly seem to be related to gcc errors in c-blosc/blosc/shuffle-avx2.c I have no idea what might be going on there and I'd appreciate a second set of eyes from someone who knows more about gcc issues.
from python-blosc.
robbmcleod
commented on July 3, 2024
Looks like the compiler doesn't know C++11 so maybe gcc is too old. 4.8.4 should be good enough though... lldiv_t seems to be bad/wrong:
http://www.cplusplus.com/reference/cstdlib/lldiv_t/
Maybe we need -std=c++11 as a compile argument?
from python-blosc.
esc
commented on July 3, 2024
Any idea where to add this?
from python-blosc.
robbmcleod
commented on July 3, 2024
I tried adding via environment variable export.
Could you also turn on the Appveyor hooks so we can see how Windows builds do? Probably 2.7 support is a non-starter but providing 3.5/3.6 wheels seems doable to me.
from python-blosc.
robbmcleod
commented on July 3, 2024
I think this must be a bug in Travis's images. There's no way lldiv shouldn't be in gcc-5.4 and it's completely random on which environment fails. Sometimes the 32-bit OS fails to find lldiv, sometimes the 64-bit, with no rhyme or reason.
from python-blosc.
robbmcleod
commented on July 3, 2024
So quick update: I forked the wheel builder (https://github.com/robbmcleod/python-blosc-wheels) so I could play with the various Travis settings, and enable Appveyor.
The Good News: Appveyor works well, and it can build all the wheels after a couple minor tweaks.
The Bad News: Travis' Linux environments are still randomly failing, even though I'm making sure to compile with gcc-5.4. I've even greped through the C++ include files on the workers and they seem to be identical on failing and passing builds:
https://travis-ci.org/robbmcleod/python-blosc-wheels/builds/302295851
So I think I'll raise an issue with Travis. The other option would be to use CircleCI for Linux and just use Travis for OSX.
from python-blosc.
itdaniher
commented on July 3, 2024
I'm interested in supporting these efforts especially on ARMv7 and ARMv8 platforms, where we use blosc at work. There aren't many commercially available hosted environments for these architectures, but I'd be glad to setup a Blosc maintainer with root accounts on Scaleways' v7 / v8 boxes, lmk!
from python-blosc.
commented on July 3, 2024
Not sure if this is the right channel for this, and I believe a PR was made a while ago, but I certainly get errors with the blosc binary (https://www.lfd.uci.edu/~gohlke/pythonlibs/) and for the life of me can't install the python blosc from source, so if there's room for a Windows binary wheel implementation please let me know.
from python-blosc.
FrancescAlted
commented on July 3, 2024
@ahendry2688 It is bad that the blosc binary at https://www.lfd.uci.edu/~gohlke/pythonlibs/ is not working correctly. Have you reported that to the maintainer?
OTOH the problems that I see with binary wheels seems related with C++ (and also possibly avoiding the use of AVX2 on machines that do not have this ISA). For Blosc2 I plan to get rid of C++ dependencies (the code will be pure C99), so that should be alleviated somehow in the future, but having a solution for binary wheel in shorter term seems not easy.
from python-blosc.
cgohlke
commented on July 3, 2024
It is bad that the blosc binary at https://www.lfd.uci.edu/~gohlke/pythonlibs/ is not working correctly.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
python-blosc version: 1.5.1
Blosc version: 1.11.3 ($Date:: 2017-03-09 #$)
Compressors available: ['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd']
Compressor library versions:
BloscLZ: 1.0.5
LZ4: 1.7.5
Snappy: 1.1.1
Zlib: 1.2.8
Zstd: 1.1.3
Python version: 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:54:40) [MSC v.1900 64 bit (AMD64)]
Platform: Windows-10-AMD64 (10.0.16299)
Processor: Intel64 Family 6 Model 79 Stepping 1, GenuineIntel
Byte-ordering: little
Detected cores: 12
Number of threads to use by default: 4
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
test_all_compressors (blosc.test.TestCodec) ... ok
test_all_filters (blosc.test.TestCodec) ... ok
test_basic_codec (blosc.test.TestCodec) ... ok
test_compress_exceptions (blosc.test.TestCodec) ... ok
test_compress_input_types (blosc.test.TestCodec) ... ok
test_compress_ptr_exceptions (blosc.test.TestCodec) ... ok
test_decompress_exceptions (blosc.test.TestCodec) ... ok
test_decompress_input_types (blosc.test.TestCodec) ... ok
test_decompress_input_types_as_bytearray (blosc.test.TestCodec) ... ok
test_decompress_ptr_exceptions (blosc.test.TestCodec) ... ok
test_decompress_releasegil (blosc.test.TestCodec) ... ok
test_get_clib (blosc.test.TestCodec) ... ok
test_no_leaks (blosc.test.TestCodec) ... ok
test_pack_array_exceptions (blosc.test.TestCodec) ... ok
test_set_nthreads_exceptions (blosc.test.TestCodec) ... ok
test_unpack_array_exceptions (blosc.test.TestCodec) ... ok
compress (blosc.toplevel)
Doctest: blosc.toplevel.compress ... ok
compress_ptr (blosc.toplevel)
Doctest: blosc.toplevel.compress_ptr ... ok
decompress (blosc.toplevel)
Doctest: blosc.toplevel.decompress ... ok
decompress_ptr (blosc.toplevel)
Doctest: blosc.toplevel.decompress_ptr ... ok
free_resources (blosc.toplevel)
Doctest: blosc.toplevel.free_resources ... ok
pack_array (blosc.toplevel)
Doctest: blosc.toplevel.pack_array ... ok
set_blocksize (blosc.toplevel)
Doctest: blosc.toplevel.set_blocksize ... ok
set_nthreads (blosc.toplevel)
Doctest: blosc.toplevel.set_nthreads ... ok
set_releasegil (blosc.toplevel)
Doctest: blosc.toplevel.set_releasegil ... ok
unpack_array (blosc.toplevel)
Doctest: blosc.toplevel.unpack_array ... ok
----------------------------------------------------------------------
Ran 26 tests in 4.094s
from python-blosc.
commented on July 3, 2024
Sorry, I meant bcolz. I did a regular pip install from the command prompt, ran bcolz.test() three times, and got errors the 3rd time:
>>> import bcolz
>>> bcolz.test()
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
bcolz version: 1.1.2
NumPy version: 1.13.3
Blosc version: 1.11.2 ($Date:: 2017-01-27 #$)
Blosc compressors: ['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd']
Numexpr version: 2.6.4
Dask version: 0.16.0
Python version: 3.5.4 (v3.5.4:3f56838, Aug 8 2017, 02:17:05) [MSC v.1900 64 bit (AMD64)]
Byte-ordering: little
Detected cores: 24
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Performing only a light (yet comprehensive) subset of the test suite.
If you want a more complete test, try passing the '-heavy' flag to this
script (or set the 'heavy' parameter in case you are using bcolz.test()
call). The whole suite will take more than 30 seconds to complete on a
relatively modern CPU and around 300 MB of RAM and 500 MB of disk
[32-bit platforms will always run significantly more lightly].
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
.................................................................................................................................................................................................................................................................................................................................................ssssssssssssssssssssssssssss.................................................................................................................ssssss.............................................................................................................................................................................................................................................................................................................................
Warning (from warnings module):
File "C:\Users\adam.hendry\AppData\Local\Programs\Python\Python35\lib\site-packages\bcolz\chunked_eval.py", line 262
"numexpr cannot handle this expression: falling back "
UserWarning: numexpr cannot handle this expression: falling back to the 'python' virtual machine. You can choose another virtual machine by using the `vm` parameter.
......................ssss.....................
Warning (from warnings module):
File "C:\Users\adam.hendry\AppData\Local\Programs\Python\Python35\lib\site-packages\bcolz\ctable.py", line 48
return re_str_split.split(str(x))
FutureWarning: split() requires a non-empty pattern match.
..............................
Warning (from warnings module):
File "C:\Users\adam.hendry\AppData\Local\Programs\Python\Python35\lib\site-packages\bcolz\ctable.py", line 693
inferred_type = pd.lib.infer_dtype(vals)
FutureWarning: pandas.lib is deprecated and will be removed in a future version.
You can access infer_dtype as pandas.api.types.infer_dtype
......................................................
Warning (from warnings module):
File "C:\Users\adam.hendry\AppData\Local\Programs\Python\Python35\lib\site-packages\bcolz\ctable.py", line 48
return re_str_split.split(str(x))
FutureWarning: split() requires a non-empty pattern match.
..................................................................................................................................................................................................................................................................................................................................................
----------------------------------------------------------------------
Ran 1270 tests in 325.611s
OK (skipped=38)
>>> bcolz.test()
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
bcolz version: 1.1.2
NumPy version: 1.13.3
Blosc version: 1.11.2 ($Date:: 2017-01-27 #$)
Blosc compressors: ['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd']
Numexpr version: 2.6.4
Dask version: 0.16.0
Python version: 3.5.4 (v3.5.4:3f56838, Aug 8 2017, 02:17:05) [MSC v.1900 64 bit (AMD64)]
Byte-ordering: little
Detected cores: 24
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Performing only a light (yet comprehensive) subset of the test suite.
If you want a more complete test, try passing the '-heavy' flag to this
script (or set the 'heavy' parameter in case you are using bcolz.test()
call). The whole suite will take more than 30 seconds to complete on a
relatively modern CPU and around 300 MB of RAM and 500 MB of disk
[32-bit platforms will always run significantly more lightly].
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
.................................................................................................................................................................................................................................................................................................................................................ssssssssssssssssssssssssssss.................................................................................................................ssssss.............................................................................................................................................................................................................................................................................................................................
Warning (from warnings module):
File "C:\Users\adam.hendry\AppData\Local\Programs\Python\Python35\lib\site-packages\bcolz\chunked_eval.py", line 262
"numexpr cannot handle this expression: falling back "
UserWarning: numexpr cannot handle this expression: falling back to the 'python' virtual machine. You can choose another virtual machine by using the `vm` parameter.
......................ssss.....................
Warning (from warnings module):
File "C:\Users\adam.hendry\AppData\Local\Programs\Python\Python35\lib\site-packages\bcolz\ctable.py", line 48
return re_str_split.split(str(x))
FutureWarning: split() requires a non-empty pattern match.
......................................................................................................................................................................................................................................................................................................................................................................................................................................
----------------------------------------------------------------------
Ran 1270 tests in 308.716s
OK (skipped=38)
>>> bcolz.test()
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
bcolz version: 1.1.2
NumPy version: 1.13.3
Blosc version: 1.11.2 ($Date:: 2017-01-27 #$)
Blosc compressors: ['blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd']
Numexpr version: 2.6.4
Dask version: 0.16.0
Python version: 3.5.4 (v3.5.4:3f56838, Aug 8 2017, 02:17:05) [MSC v.1900 64 bit (AMD64)]
Byte-ordering: little
Detected cores: 24
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Performing only a light (yet comprehensive) subset of the test suite.
If you want a more complete test, try passing the '-heavy' flag to this
script (or set the 'heavy' parameter in case you are using bcolz.test()
call). The whole suite will take more than 30 seconds to complete on a
relatively modern CPU and around 300 MB of RAM and 500 MB of disk
[32-bit platforms will always run significantly more lightly].
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
.........................................................E.........................................E.............................................................................................................................................................................................................................................ssssssssssssssssssssssssssss.................................................................................................................ssssss.............................................................................................................................................................................................................................................................................................................................
Warning (from warnings module):
File "C:\Users\adam.hendry\AppData\Local\Programs\Python\Python35\lib\site-packages\bcolz\chunked_eval.py", line 262
"numexpr cannot handle this expression: falling back "
UserWarning: numexpr cannot handle this expression: falling back to the 'python' virtual machine. You can choose another virtual machine by using the `vm` parameter.
......................ssss.....................
Warning (from warnings module):
File "C:\Users\adam.hendry\AppData\Local\Programs\Python\Python35\lib\site-packages\bcolz\ctable.py", line 48
return re_str_split.split(str(x))
FutureWarning: split() requires a non-empty pattern match.
......................................................................................................................................................................................................................................................................................................................................................................................................................................
======================================================================
ERROR: test03 (test_carray.appendDiskTest)
Testing `append()` method (large append)
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\Users\adam.hendry\AppData\Local\Programs\Python\Python35\lib\site-packages\bcolz\tests\common.py", line 54, in tearDown
shutil.rmtree(dir_)
File "C:\Users\adam.hendry\AppData\Local\Programs\Python\Python35\lib\shutil.py", line 494, in rmtree
return _rmtree_unsafe(path, onerror)
File "C:\Users\adam.hendry\AppData\Local\Programs\Python\Python35\lib\shutil.py", line 384, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "C:\Users\adam.hendry\AppData\Local\Programs\Python\Python35\lib\shutil.py", line 393, in _rmtree_unsafe
onerror(os.rmdir, path, sys.exc_info())
File "C:\Users\adam.hendry\AppData\Local\Programs\Python\Python35\lib\shutil.py", line 391, in _rmtree_unsafe
os.rmdir(path)
OSError: [WinError 145] The directory is not empty: 'C:\\Users\\ADAM~1.HEN\\AppData\\Local\\Temp\\bcolz-appendDiskTest5cph3gfv\\meta'
======================================================================
ERROR: test01c (test_carray.compressorsDiskTest)
Testing all available compressors in big arrays (context)
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\Users\adam.hendry\AppData\Local\Programs\Python\Python35\lib\site-packages\bcolz\tests\test_carray.py", line 2217, in test01c
common.remove_tree(self.rootdir)
File "C:\Users\adam.hendry\AppData\Local\Programs\Python\Python35\lib\site-packages\bcolz\tests\common.py", line 34, in remove_tree
shutil.rmtree(dir_)
File "C:\Users\adam.hendry\AppData\Local\Programs\Python\Python35\lib\shutil.py", line 494, in rmtree
return _rmtree_unsafe(path, onerror)
File "C:\Users\adam.hendry\AppData\Local\Programs\Python\Python35\lib\shutil.py", line 384, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "C:\Users\adam.hendry\AppData\Local\Programs\Python\Python35\lib\shutil.py", line 393, in _rmtree_unsafe
onerror(os.rmdir, path, sys.exc_info())
File "C:\Users\adam.hendry\AppData\Local\Programs\Python\Python35\lib\shutil.py", line 391, in _rmtree_unsafe
os.rmdir(path)
OSError: [WinError 145] The directory is not empty: 'C:\\Users\\ADAM~1.HEN\\AppData\\Local\\Temp\\bcolz-compressorsDiskTestxkuxp275\\meta'
----------------------------------------------------------------------
Ran 1270 tests in 304.416s
FAILED (errors=2, skipped=38)
from python-blosc.
commented on July 3, 2024
Blosc I ran in a loop 10 times and got no errors
from python-blosc.
commented on July 3, 2024
Errors above in bcolz also happened with the installed binary wheel from Gohlke too (https://www.lfd.uci.edu/~gohlke/pythonlibs/). I will let @cgohlke know.
from python-blosc.
FrancescAlted
commented on July 3, 2024
This actually belongs to bcolz: created a new ticket at: Blosc/bcolz#363
from python-blosc.
FrancescAlted
commented on July 3, 2024
Oops, I did not intend closing this.
from python-blosc.
commented on July 3, 2024
Thank you @FrancescAlted. Apologies for putting the bcolz issue in the blosc thread.
from python-blosc.
robbmcleod
commented on July 3, 2024
I finally got around to raising an issue with Travis, we'll see where it leads:
from python-blosc.
esc
commented on July 3, 2024
I'm still interested in this, is there any way I can help out? python-blosc and hence also bloscpack still depends on a compiler and python-dev etc.. Which is causing installation related issues sometimes.
from python-blosc.
robbmcleod
commented on July 3, 2024
My gut feeling is that something is going wrong network-wise when the Travis workers try to load the docker images. The approach I would take would be to SSH into the worker nodes and try and catch a broken environment by stepping through the build.
https://docs.travis-ci.com/user/running-build-in-debug-mode/
The alternative would be to try and use different docker images, but that seems way too much work for me.
from python-blosc.
matthew-brett
commented on July 3, 2024
Sorry not to investigate more, but do I understand right, that sometimes the manylinux docker image has the right headers, and sometimes not?
There are some plumper manylinux images you could use - https://github.com/dockcross/dockcross . You can change the docker image in the multibuild framework by setting the DOCKER_IMAGE environment variable in .travis.yml to give the URL to the image you want.
from python-blosc.
esc
commented on July 3, 2024
Hi, can I ping you guys on this at all? Is there anything I can do to help?
from python-blosc.
esc
commented on July 3, 2024
@robbmcleod I tried the changes from your fork last night and can confirm that Appveyor is building fine, but Travis is still causing issues, will need to drill into the build.
from python-blosc.
robbmcleod
commented on July 3, 2024
Yeah I don't really have the time or energy to work on this sorry. From what I recall, there were compile bugs that were inconsistent. My feeling that sometimes the Docker images weren't loading properly.
Probably what is required is to ask Travis to enable debug mode and then SSH into the workers that puke.
from python-blosc.
thewtex
commented on July 3, 2024
Hi there,
I discovered this package and was very impressed! I would like to use it as a dependency, but the lack of binary packages prevents this. If we added scikit-build configuration to help build the wheels and scikit-ci configuration to build per platform, would they be integrated and the packages pushed to PyPI?
CC: @jcfr
from python-blosc.
esc
commented on July 3, 2024
@thewtex I learned today, that an up to date binary binary package might be available via conda. Perhaps you could give that a try python-blosc and see if it gives you some milage?
from python-blosc.
matthew-brett
commented on July 3, 2024
Can someone give me a little summary about why the blosc-wheels build don't work at the moment?
from python-blosc.
esc
commented on July 3, 2024
@matthew-brett, sure, we never got them to work in the first place because we never managed to get the Travis builds to work.
from python-blosc.
matthew-brett
commented on July 3, 2024
You mean you can't build the distribution at all on travis? Or you can't build the wheels? Is there some list of problems or link to build error logs?
from python-blosc.
esc
commented on July 3, 2024
The regular integration tests run fine and we can and do build those on Travis. But for building the wheels on Travis we get a variety of different compilation issues. My most recent attempts are here:
https://github.com/esc/python-blosc-wheels
And the corresponding Travis-builds are here:
https://travis-ci.org/esc/python-blosc-wheels
from python-blosc.
esc
commented on July 3, 2024
@oscargm98 @FrancescAlted I see some work was done on the binary wheels, can this issue be closed now?
from python-blosc.
FrancescAlted
commented on July 3, 2024
Sure @esc . Thanks for the reminder!
from python-blosc.
Related Issues (20)
- Issues decompressing bytes from files HOT 1
- Replace obsolete `popen2` HOT 1
- Properly identify vendored `cpuinfo.py` version
- Blosc_ROOT cmake warning: Policy CMP0074 is not set HOT 2
- "RuntimeError: Cannot decompress" for a compressed sequence of more than 7240 zero bytes HOT 1
- Very bad compression on short inputs 1-127 bytes long HOT 5
- “python_requires” should be set with “>=3.6”, as blosc 1.10.6 is not compatible with all Python versions. HOT 2
- wrong setuptools build command
- Concatenate two blosc compressed bytes objects HOT 2
- LICENSES/BLOSC.txt HOT 4
- Rename default branch HOT 1
- Update pypi with latest blosc version HOT 3
- Wheel for Python 3.10 and Python 3.11 HOT 3
- Cannot install blosc 1.11.0 on apple M1 machine HOT 3
- decompress in fore-end HOT 1
- README link to python-blosc2 seems useful HOT 1
- __pack_tensor__ must be made portable and not depend on Python HOT 2
- __pack_tensor__ should be in the beginning of the file to avoid seeking the whole file HOT 2
- Python 3.12 compatibility HOT 6
- Numpy 2 compatibility
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from python-blosc.