juliaio / blosc.jl Goto Github PK

View Code? Open in Web Editor NEW

29.0 14.0 28.0 100 KB

Blosc compression for the Julia language

License: Other

Julia 100.00%

julia compression blosc

blosc.jl's Introduction

The Blosc Module for Julia

This module provides fast lossless compression for the Julia language by interfacing the Blosc Library.

Note that Blosc is currently limited to 32-bit buffer sizes. Blosc does run just fine on 64-bit systems; it just can't compress arrays bigger than 2GB. Note also that this limitation does not affect the use of Blosc compression for HDF5, since HDF5 breaks up large arrays into small chunks before compressing them. So, don't worry about this if you are just using Blosc within the HDF5 package.

Installation

To install, simply run Pkg.add("Blosc") in Julia. Precompiled binaries are provided for Mac and Windows systems, while on other systems the Blosc library will be downloaded and compiled.

Usage

The functions provided are:

compress(src::Array{T}; level=5, shuffle=true, itemsize=sizeof(T)): returns a Vector{UInt8} consisting of src in compressed form. level is the compression level (between 0=no compression and 9=max), shuffle indicates whether to use Blosc's shuffling preconditioner, which is optimized for arrays of binary items of size itemsize.
compress!(dest::Vector{UInt8}, src; ...): as compress, but uses a pre-allocated destination buffer dest. Returns the size (in bytes) of the compressed data, or 0 if the buffer was too small.
decompress(T::Type, src::Vector{UInt8}): return the compressed buffer src as an array of element type T.
decompress!(dest::Vector{T}, src::Vector{UInt8}): like decompress, but uses a pre-allocated destination buffer, which is resized as needed to store the decompressed data.
Blosc.set_num_threads(n=CPU_CORES): tells Blosc to use n threads (initially 1).
Blosc.compressors(): returns an array of strings for the available compression algorithms. (The blosclz compressor is the default.)
Blosc.set_compressor(s::AbstractString): set the current compression algorithm

Author

This module was written by Steven G. Johnson and Jake Bolewski (who had independently written his own Blosc.jl package which is now merged with this one).

blosc.jl's People

Contributors

Stargazers

Watchers

blosc.jl's Issues

can't install Blosc under julia 0.4.6, linux 64-bit Centos 6.8

I'm trying to port some of my Julia work from Windows to a Linux server, but I was not able to install the JLD package. Apparently the problem with JLD is that the Blosc library can't be installed. Here is the trace.

Thanks,
Steve Vavasis

@cpu135[92]% uname -a
Linux cpu135 2.6.32-642.4.2.el6.x86_64 #1 SMP Tue Aug 23 19:58:13 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
@cpu135[93]% cat /etc/issue
CentOS release 6.8 (Final)
Kernel \r on an \m
@cpu135[97]% ./julia
_
_ _ ()_ | A fresh approach to technical computing
() | () () | Documentation: http://docs.julialang.org
_ _ | | __ _ | Type "?help" for help.
| | | | | | |/ ` | |
| | || | | | (| | | Version 0.4.6 (2016-06-19 17:16 UTC)
/ |_'|||__'| | Official http://julialang.org/ release
|__/ | x86_64-unknown-linux-gnu

julia> Pkg.add("Blosc")
INFO: Installing Blosc v0.1.7
INFO: Building Blosc

gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now
================================[ ERROR: Blosc ]================================

LoadError: failed process: Process(tar xzf c-blosc-1.8.1.tar.gz --directory=., ProcessExited(2)) [2]
while loading /u4/vavasis/.julia/v0.4/Blosc/deps/build.jl, in expression starting on line 20

================================[ BUILD ERRORS ]================================

WARNING: Blosc had build errors.

packages with build errors remain installed in /u4/vavasis/.julia/v0.4
build the package(s) and all dependencies with Pkg.build("Blosc")
build a single package by running its deps/build.jl script

INFO: Package database updated

julia>

TranscodingStreams API

Hi,

I'm currently working on bringing better compression to [JLD2.(https://github.com/JuliaIO/JLD2.jl).
Most of the other compression libraries seem to support the TranscodingStreams API and it appears to be straight forward to utilize that for JLD2.

At this time Blosc is not compatible to TranscodingStreams but it would be much easier to enable it within JLD2 if it were.
Would it be possible to add the API functions? Has this been discussed before?

Sys.CPU_CORES not defined in recent julia version

https://github.com/stevengj/Blosc.jl/blob/6a5756023fa9e972cffa4ad3d30df66ea30798e5/src/Blosc.jl#L200

Not sure what the alternative is now? Sys.CPU_THREADS?

Release a new version

It would be nice to have a new release, featuring #71.

Sub-optimal binaries for Blosc library

In the attached benchmark (sum-bench.jl.zip), a fresh install of the Blosc package on a Mac OSX box leaves to these timings:

$ julia sum-bench.jl 
  51.504 ms (0 allocations: 0 bytes)
  51.591 ms (3 allocations: 48 bytes)
  92.988 ms (754 allocations: 11.80 KiB)
  407.834 ms (754 allocations: 11.80 KiB)
  379.742 ms (750 allocations: 11.72 KiB)
  378.752 ms (0 allocations: 0 bytes)

When replaced the Blosc library by one that is compiled natively (via cmake) with:

$ cp ~/blosc/c-blosc/build/blosc/libblosc.1.14.3.dylib ~/.julia/v0.6/Blosc/deps/usr/lib/libblosc.1.14.3.dylib

and re-running the benchmark, I get this:

$ julia sum-bench.jl 
  50.631 ms (0 allocations: 0 bytes)
  51.209 ms (3 allocations: 48 bytes)
  92.434 ms (754 allocations: 11.80 KiB)
  116.036 ms (754 allocations: 11.80 KiB)
  114.799 ms (750 allocations: 11.72 KiB)
  113.008 ms (0 allocations: 0 bytes)

The last three lines in the timings represent benchmarks using the Blosc libraries. As can be seen, a native Blosc library performs around 3x faster than the binary that is being distributed.

`Blosc.compress(rand(UInt8,2^31-17))` causes Segmentation fault

I get a segmentation fault when compressing 2^31-17 random bytes. Is there a maximum size for compress?

(jl_zgpc5j) pkg> add Blosc
   Resolving package versions...
    Updating `/tmp/jl_zgpc5j/Project.toml`
  [a74b3585] + Blosc v0.7.3
    Updating `/tmp/jl_zgpc5j/Manifest.toml`
  [a74b3585] + Blosc v0.7.3
  [692b3bcd] + JLLWrappers v1.5.0
  [21216c6a] + Preferences v1.4.3
  [0b7ba130] + Blosc_jll v1.21.5+0
  [5ced341a] + Lz4_jll v1.9.4+0
  [3161d3a3] + Zstd_jll v1.5.6+0
  [56f22d72] + Artifacts
  [ade2ca70] + Dates
  [8f399da3] + Libdl
  [de0858da] + Printf
  [fa267f1f] + TOML v1.0.3
  [4ec0a83e] + Unicode
  [83775a58] + Zlib_jll v1.2.13+1

julia> using Blosc

julia> Blosc.compress(rand(UInt8,2^31-17));

[413064] signal (11.1): Segmentation fault
in expression starting at REPL[4]:1
fastcopy at /home/nathan/.julia/artifacts/7969c83077619d040d72f0549771cd9506c5c1bd/lib/libblosc.so (unknown line)
blosc_c at /home/nathan/.julia/artifacts/7969c83077619d040d72f0549771cd9506c5c1bd/lib/libblosc.so (unknown line)
do_job at /home/nathan/.julia/artifacts/7969c83077619d040d72f0549771cd9506c5c1bd/lib/libblosc.so (unknown line)
blosc_compress at /home/nathan/.julia/artifacts/7969c83077619d040d72f0549771cd9506c5c1bd/lib/libblosc.so (unknown line)
blosc_compress at /home/nathan/.julia/packages/Blosc/jk4Np/src/Blosc.jl:36 [inlined]
#compress!#3 at /home/nathan/.julia/packages/Blosc/jk4Np/src/Blosc.jl:96
compress! at /home/nathan/.julia/packages/Blosc/jk4Np/src/Blosc.jl:74 [inlined]
#compress#6 at /home/nathan/.julia/packages/Blosc/jk4Np/src/Blosc.jl:111
compress at /home/nathan/.julia/packages/Blosc/jk4Np/src/Blosc.jl:109 [inlined]
#compress#7 at /home/nathan/.julia/packages/Blosc/jk4Np/src/Blosc.jl:117 [inlined]
compress at /home/nathan/.julia/packages/Blosc/jk4Np/src/Blosc.jl:115
unknown function (ip: 0x7cf5c31200e5)
_jl_invoke at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/gf.c:2895 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/gf.c:3077
jl_apply at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
do_call at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/interpreter.c:126
eval_value at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/interpreter.c:223
eval_stmt_value at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/interpreter.c:174 [inlined]
eval_body at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/interpreter.c:617
jl_interpret_toplevel_thunk at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/interpreter.c:775
jl_toplevel_eval_flex at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/toplevel.c:934
jl_toplevel_eval_flex at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/toplevel.c:877
jl_toplevel_eval_flex at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/toplevel.c:877
jl_toplevel_eval_flex at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/toplevel.c:877
jl_toplevel_eval_flex at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/toplevel.c:877
ijl_toplevel_eval_in at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/toplevel.c:985
eval at ./boot.jl:385 [inlined]
eval_user_input at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:150
repl_backend_loop at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:246
#start_repl_backend#46 at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:231
start_repl_backend at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:228
_jl_invoke at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/gf.c:2895 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/gf.c:3077
#run_repl#59 at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:389
run_repl at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:375
jfptr_run_repl_91734.1 at /home/nathan/.julia/juliaup/julia-1.10.3+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/gf.c:2895 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/gf.c:3077
#1013 at ./client.jl:432
jfptr_YY.1013_82700.1 at /home/nathan/.julia/juliaup/julia-1.10.3+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/gf.c:2895 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/gf.c:3077
jl_apply at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
jl_f__call_latest at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/builtins.c:812
#invokelatest#2 at ./essentials.jl:892 [inlined]
invokelatest at ./essentials.jl:889 [inlined]
run_main_repl at ./client.jl:416
exec_options at ./client.jl:333
_start at ./client.jl:552
jfptr__start_82726.1 at /home/nathan/.julia/juliaup/julia-1.10.3+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/gf.c:2895 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/gf.c:3077
jl_apply at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
true_main at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/jlapi.c:582
jl_repl_entrypoint at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/src/jlapi.c:731
main at /cache/build/builder-amdci4-2/julialang/julia-release-1-dot-10/cli/loader_exe.c:58
unknown function (ip: 0x7cf5c4229d8f)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x4010b8)
Allocations: 3723792 (Pool: 3719955; Big: 3837); GC: 7
Segmentation fault (core dumped)

julia> versioninfo()
Julia Version 1.10.3
Commit 0b4590a5507 (2024-04-30 10:59 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 16 × AMD Ryzen 7 7800X3D 8-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 16 virtual cores)

Tag new release

Last release was on Aug 18, 2016 .

move to JuliaIO?

Since HDF5 and JLD depend on this, although Blosc is not really "IO" specifically. @tkelman, @timholy?

GC.@preserve the results of cbuffer_sizes

For reasons unclear to me, it may be necessary to GC.@preserve the results of Blosc.cbuffer_sizes, otherwise the original buffer may become corrupted. This occurs starting with Julia 1.8 in development.

JuliaIO/HDF5.jl#880 (comment)

Problems installing HDF5 as test dependency due to Blosc

I'm having trouble with HDF5 as a test dependency on Julia v1.3, due to Blosc nailing down Compat or something like that:

https://travis-ci.com/github/bat/BAT.jl/jobs/296129424

I don't quite see what's going on here, since the tests pass with Julia v1.4 - could it be because Blosc doesn't have a Project.toml yet?

can't install blosc in latest nightly

I downloaded the latest nightly for 0.7-DEV, Ubuntu Linux 17.10, and was not able to install JLD because Blosc installation failed. See below for the error message.

julia> Pkg.add("Blosc")
[ Info: Installing BinDeps v0.8.6
[ Info: Installing Blosc v0.3.0
[ Info: Installing SHA v0.5.6
[ Info: Installing URIParser v0.3.0
[ Info: Building Blosc
WARNING: importing deprecated binding Base.Libdl into __anon__.
WARNING: Base.Libdl is deprecated, run `using Libdl` instead
  likely near /home/vavasis/.julia/v0.7/Blosc/deps/build.jl:6
┌ Warning: `is_windows` is deprecated, use `Sys.iswindows` instead.
│   caller = top-level scope at build.jl:10
└ @ Core build.jl:10
┌ Warning: `is_apple` is deprecated, use `Sys.isapple` instead.
│   caller = top-level scope at build.jl:12
└ @ Core build.jl:12
Compiling libblosc...
┌ Error: ------------------------------------------------------------
│ # Build failed for Blosc
│   exception =
│    LoadError: could not spawn `make -f ../../make.blosc LIB=../../libblosc.so`: no such file or directory (ENOENT)
│    Stacktrace:
│     [1] _jl_spawn(::String, ::Array{String,1}, ::Cmd, ::Tuple{RawFD,RawFD,RawFD}) at ./process.jl:370
│     [2] (::getfield(Base, Symbol("##519#520")){Cmd})(::Tuple{RawFD,RawFD,RawFD}) at ./process.jl:514
│     [3] setup_stdio(::getfield(Base, Symbol("##519#520")){Cmd}, ::Tuple{RawFD,RawFD,RawFD}) at ./process.jl:493
│     [4] #spawn#518(::Nothing, ::Function, ::Cmd, ::Tuple{RawFD,RawFD,RawFD}) at ./process.jl:511
│     [5] spawn at ./process.jl:507 [inlined]
│     [6] run(::Cmd) at ./process.jl:654
│     [7] (::getfield(Main.__anon__, Symbol("##1#3")))() at /home/vavasis/.julia/v0.7/Blosc/deps/build.jl:23
│     [8] cd(::getfield(Main.__anon__, Symbol("##1#3")), ::String) at ./file.jl:70
│     [9] top-level scope at /home/vavasis/.julia/v0.7/Blosc/deps/build.jl:21
│     [10] include at ./boot.jl:295 [inlined]
│     [11] include_relative(::Module, ::String) at ./loading.jl:1060
│     [12] include at ./sysimg.jl:27 [inlined]
│     [13] include(::String) at ./loading.jl:1094
│     [14] top-level scope
│     [15] eval at ./boot.jl:298 [inlined]
│     [16] eval at ./sysimg.jl:72 [inlined]
│     [17] evalfile(::String, ::Array{String,1}) at ./loading.jl:1089 (repeats 2 times)
│     [18] #2 at ./none:15 [inlined]
│     [19] cd(::getfield(, Symbol("##2#4")){String}, ::String) at ./file.jl:70
│     [20] (::getfield(, Symbol("##1#3")))(::IOStream) at ./none:14
│     [21] #open#315(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::getfield(, Symbol("##1#3")), ::String, ::Vararg{String,N} where N) at ./iostream.jl:369
│     [22] open(::Function, ::String, ::String) at ./iostream.jl:367
│     [23] top-level scope
│     [24] eval at ./boot.jl:298 [inlined]
│     [25] eval(::Module, ::Expr) at ./sysimg.jl:72
│     [26] exec_options(::Base.JLOptions) at ./client.jl:309
│     [27] _start() at ./client.jl:448
│    in expression starting at /home/vavasis/.julia/v0.7/Blosc/deps/build.jl:21
└ @ Main none:18
┌ Warning: ------------------------------------------------------------
│ # Build error summary
│ 
│ Blosc had build errors.
│ 
│  - packages with build errors remain installed in /home/vavasis/.julia/v0.7
│  - build the package(s) and all dependencies with `Pkg.build("Blosc")`
│  - build a single package by running its `deps/build.jl` script
└ @ Pkg.Entry entry.jl:651
[ Info: Package database updated

julia>

blosc tests fail in windows 11

(@v1.9) pkg> test Blosc
     Testing Blosc
      Status `C:\Users\arthurb\AppData\Local\Temp\jl_kMcaTe\Project.toml`
  [a74b3585] Blosc v0.7.3
  [0b7ba130] Blosc_jll v1.21.4+0
  [8dfed614] Test `@stdlib/Test`
      Status `C:\Users\arthurb\AppData\Local\Temp\jl_kMcaTe\Manifest.toml`
  [a74b3585] Blosc v0.7.3
  [692b3bcd] JLLWrappers v1.5.0
  [21216c6a] Preferences v1.4.1
  [0b7ba130] Blosc_jll v1.21.4+0
  [5ced341a] Lz4_jll v1.9.4+0
  [3161d3a3] Zstd_jll v1.5.5+0
  [56f22d72] Artifacts `@stdlib/Artifacts`
  [2a0f44e3] Base64 `@stdlib/Base64`
  [ade2ca70] Dates `@stdlib/Dates`
  [b77e0a4c] InteractiveUtils `@stdlib/InteractiveUtils`
  [8f399da3] Libdl `@stdlib/Libdl`
  [56ddb016] Logging `@stdlib/Logging`
  [d6f4376e] Markdown `@stdlib/Markdown`
  [de0858da] Printf `@stdlib/Printf`
  [9a3f8284] Random `@stdlib/Random`
  [ea8e919c] SHA v0.7.0 `@stdlib/SHA`
  [9e88b42a] Serialization `@stdlib/Serialization`
  [fa267f1f] TOML v1.0.3 `@stdlib/TOML`
  [8dfed614] Test `@stdlib/Test`
  [4ec0a83e] Unicode `@stdlib/Unicode`
  [83775a58] Zlib_jll v1.2.13+0 `@stdlib/Zlib_jll`
Precompiling project...
  6 dependencies successfully precompiled in 11 seconds. 1 already precompiled.
     Testing Running tests...

Invalid instruction at 000000006800d83e: 0xc5, 0xfa, 0x7e, 0x54, 0x24, 0x28, 0x48, 0x89, 0x54, 0x24, 0x28, 0xc5, 0xfa, 0x7e, 0x5c

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ILLEGAL_INSTRUCTION at 0x6800d83e -- .text at C:\Users\arthurb\.julia\artifacts\e48a0bacd2ce6a108c7edc024073f889d920d522\bin\libblosc.dll (unknown line)
in expression starting at C:\Users\arthurb\.julia\packages\Blosc\jk4Np\test\runtests.jl:27
.text at C:\Users\arthurb\.julia\artifacts\e48a0bacd2ce6a108c7edc024073f889d920d522\bin\libblosc.dll (unknown line)
pthread_once at /workspace/srcdir/mingw-w64-v7.0.0/mingw-w64-libraries/winpthreads/src\thread.c:745
blosc_internal_shuffle at C:\Users\arthurb\.julia\artifacts\e48a0bacd2ce6a108c7edc024073f889d920d522\bin\libblosc.dll (unknown line)
blosc_c at C:\Users\arthurb\.julia\artifacts\e48a0bacd2ce6a108c7edc024073f889d920d522\bin\libblosc.dll (unknown line)
do_job at C:\Users\arthurb\.julia\artifacts\e48a0bacd2ce6a108c7edc024073f889d920d522\bin\libblosc.dll (unknown line)
blosc_compress at C:\Users\arthurb\.julia\artifacts\e48a0bacd2ce6a108c7edc024073f889d920d522\bin\libblosc.dll (unknown line)
blosc_compress at C:\Users\arthurb\.julia\packages\Blosc\jk4Np\src\Blosc.jl:36 [inlined]
#compress!#3 at C:\Users\arthurb\.julia\packages\Blosc\jk4Np\src\Blosc.jl:96
compress! at C:\Users\arthurb\.julia\packages\Blosc\jk4Np\src\Blosc.jl:74 [inlined]
#compress#6 at C:\Users\arthurb\.julia\packages\Blosc\jk4Np\src\Blosc.jl:111
compress at C:\Users\arthurb\.julia\packages\Blosc\jk4Np\src\Blosc.jl:109 [inlined]
#compress#7 at C:\Users\arthurb\.julia\packages\Blosc\jk4Np\src\Blosc.jl:117 [inlined]
compress at C:\Users\arthurb\.julia\packages\Blosc\jk4Np\src\Blosc.jl:115
unknown function (ip: 0000029e1d7163e6)
jl_apply at C:/workdir/src\julia.h:1880 [inlined]
do_call at C:/workdir/src\interpreter.c:126
eval_value at C:/workdir/src\interpreter.c:226
eval_stmt_value at C:/workdir/src\interpreter.c:177 [inlined]
eval_body at C:/workdir/src\interpreter.c:624
eval_body at C:/workdir/src\interpreter.c:533
jl_interpret_toplevel_thunk at C:/workdir/src\interpreter.c:762
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:912
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:856
ijl_toplevel_eval at C:/workdir/src\toplevel.c:921 [inlined]
ijl_toplevel_eval_in at C:/workdir/src\toplevel.c:971
eval at .\boot.jl:370 [inlined]
include_string at .\loading.jl:1903
_include at .\loading.jl:1963
include at .\client.jl:478
unknown function (ip: 0000029e1d710086)
jl_apply at C:/workdir/src\julia.h:1880 [inlined]
do_call at C:/workdir/src\interpreter.c:126
eval_value at C:/workdir/src\interpreter.c:226
eval_stmt_value at C:/workdir/src\interpreter.c:177 [inlined]
eval_body at C:/workdir/src\interpreter.c:624
jl_interpret_toplevel_thunk at C:/workdir/src\interpreter.c:762
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:912
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:856
ijl_toplevel_eval at C:/workdir/src\toplevel.c:921 [inlined]
ijl_toplevel_eval_in at C:/workdir/src\toplevel.c:971
eval at .\boot.jl:370 [inlined]
exec_options at .\client.jl:280
_start at .\client.jl:522
jfptr__start_29544 at C:\Users\arthurb\.julia\juliaup\julia-1.9.3+0.x64.w64.mingw32\lib\julia\sys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:1880 [inlined]
true_main at C:/workdir/src\jlapi.c:573
jl_repl_entrypoint at C:/workdir/src\jlapi.c:717
mainCRTStartup at C:/workdir/cli\loader_exe.c:59
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
Allocations: 2998 (Pool: 2986; Big: 12); GC: 0
ERROR: Package Blosc errored during testing

Allow compression of Base.FastContiguousSubArray

Currently Blosc.jl allows for compression of DenseArrays that are contiguous. Besides being contiguous, the main requirement for an AbstractArray is the applicability of pointer.

Base.FastContiguousSubArray satisfies contiguity and has method pointer.

julia> Base.FastContiguousSubArray
SubArray{T,N,P,I,true} where I<:Union{Tuple{Vararg{Real,N} where N}, Tuple{AbstractUnitRange,Vararg{Any,N} where N}} where P where N where T

julia> hasmethod(pointer, (Base.FastContiguousSubArray,))
true

SIMD.FastContiguousArray also satisfies contiguity and has method pointer

julia> using SIMD

julia> SIMD.FastContiguousArray
Union{DenseArray{T,N}, Base.ReinterpretArray{T,N,T2,A} where A<:Union{DenseArray{T2,N}, SubArray{T2,N,P,I,true} where I<:Union{Tuple{Vararg{Real,N} where N}, Tuple{AbstractUnitRange,Vararg{Any,N} where N}} where P} where T2, SubArray{T,N,P,I,true} where I<:Union{Tuple{Vararg{Real,N} where N}, Tuple{AbstractUnitRange,Vararg{Any,N} where N}} where P} where N where T

julia> hasmethod(pointer, (SIMD.FastContiguousArray,))
true

https://github.com/eschnett/SIMD.jl/blob/master/src/arrayops.jl#L40-L42

Could we widen the compress, compress!, decompress, and decompress! methods to include these types, perhaps in place of DenseArray?

Build failures on master and windows

See e.g. https://ci.appveyor.com/project/StevenGJohnson/blosc-jl

Build fails on Julia v0.6

Can you please confirm the issue?

https://travis-ci.org/juliohm/ImageQuilting.jl/jobs/192579809

Blosc2 support?

Hi,

What are your thoughts on supporting Blosc2?

Thank you,

Rana

Update c-blosc to version 1.7.1

The release notes are at https://github.com/Blosc/c-blosc/blob/master/RELEASE_NOTES.rst#changes-from-170-to-171.

SSL certificate failures on Travis

I'm seeing occasional Travis failures for Julia 0.6 with the error:

WARNING: Blosc had build errors.
 - packages with build errors remain installed in /home/travis/.julia/v0.6
 - build the package(s) and all dependencies with `Pkg.build("Blosc")`
 - build a single package by running its `deps/build.jl` script
================================================================================
INFO: Building Blosc
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (60) SSL certificate problem, verify that the CA cert is OK. Details:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
More details here: http://curl.haxx.se/docs/sslcerts.html

@staticfloat, @tkelman, is there anything that can be done about this?

Cannot update to Blosc 0.2.0 on Windows

Blosc was working on Windows machines until recently, but when I ran Pkg.update(), it failed with the following errors:

INFO: Building Blosc
Exception calling "DownloadFile" with "2" argument(s): "An exception occurred during a WebClient request."
At line:1 char:1
+ (new-object net.webclient).DownloadFile("https://bintray.com/artifact ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : WebException

====================================================[ ERROR: Blosc ]====================================================

LoadError: failed process: Process(`'C:\Windows\System32\WindowsPowerShell\v1.0\powershell' -NoProfile -Command '(new-object net.webclient).DownloadFile("https://bintray.com/artifact/download/julialang/generic/libblosc64-1.11.1.dll", "libblosc.dll")'`, ProcessExited(1)) [1]
while loading C:\Users\xxx\.julia\v0.5\Blosc\deps\build.jl, in expression starting on line 11

========================================================================================================================

====================================================[ BUILD ERRORS ]====================================================

WARNING: Blosc had build errors.

 - packages with build errors remain installed in C:\Users\xxx\.julia\v0.5
 - build the package(s) and all dependencies with `Pkg.build("Blosc")`
 - build a single package by running its `deps/build.jl` script

========================================================================================================================

I didn't have any problems updating Blosc on my Mac OS X and Ubuntu/Linux machines. So, this is certainly a Windows-specific problem.
Thanks a lot for your help!
BVPs

use partr threads backend

Would be nice to use partr for multi-threading, so that multi-threaded Blosc becomes composable with other Julia multi-threaded code. We might be able to do something similar to FFTW/fftw3#175

Tag new release?

The latest release still has OS_NAME depwarns. Could you tag a new release please?

Build fails on Julia v0.3.3

I ran Pkg.update(), and now the Blosc package is breaking everything.

julia> Pkg.build("Blosc")
INFO: Building Blosc
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (77) error setting certificate verify locations:
  CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
=======================================================[ ERROR: Blosc ]=======================================================

failed process: Process(`curl -o c-blosc-1.5.0.tar.gz -L https://github.com/Blosc/c-blosc/archive/v1.5.0.tar.gz`, ProcessExited(77)) [77]
while loading /home/dss/.julia/v0.3/Blosc/deps/build.jl, in expression starting on line 16

==============================================================================================================================

=======================================================[ BUILD ERRORS ]=======================================================

WARNING: Blosc had build errors.

 - packages with build errors remain installed in /home/dss/.julia/v0.3
 - build a package and all its dependencies with `Pkg.build(pkg)`
 - build a single package by running its `deps/build.jl` script

==============================================================================================================================

julia> versioninfo()
Julia Version 0.3.3
Commit b24213b* (2014-11-23 20:19 UTC)
Platform Info:
  System: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Xeon(R) CPU E5-2665 0 @ 2.40GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Sandybridge)
  LAPACK: libopenblas
  LIBM: libopenlibm
  LLVM: libLLVM-3.3

cc1: error: unrecognized command line option ‘-mavx2’

I tried to restart the build, but the error is persistent on Julia nightly and Linux: https://travis-ci.org/juliohm/ImageQuilting.jl/jobs/175836611#L240

libblosc compile error on ARM

I have tried to install the MAT package but unfortunately it failed due to an error when compiling one of its dependencies, Blosc. It looks like the make command is not using the proper arm flags and instead using the x86.
Blosc recently received official ARM support so compiling it in ARM should not be a problem.

Julia version: 0.4.0-dev+6443 / Commit f0328c8* / arm-linux-gnueabihf

Julia output:

Compiling libblosc...
gcc -I. -DSHUFFLE_SSE2_ENABLED -DBLOSC_SHARED_LIBRARY -fPIC -O3 -pthread -msse2 -c -o bitshuffle-sse2.o bitshuffle-sse2.c
gcc: error: unrecognized command line option ‘-msse2’
../../make.blosc:38: recipe for target 'bitshuffle-sse2.o' failed
make: *** [bitshuffle-sse2.o] Error 1
==============================================================[ ERROR: Blosc ]===============================================================

LoadError: failed process: Process(`make -f ../../make.blosc HAVE_AVX=0 LIB=../../libblosc.so`, ProcessExited(2)) [2]
while loading /root/.julia/v0.4/Blosc/deps/build.jl, in expression starting on line 21

Pkg.add("Blosc.jl") fails due to curl

julia> Pkg.add("Blosc")
INFO: Installing BinDeps v0.4.5
INFO: Installing Blosc v0.1.7
INFO: Installing SHA v0.2.1
INFO: Installing URIParser v0.1.6
INFO: Building Blosc
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (77) error setting certificate verify locations:
  CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
================================[ ERROR: Blosc ]================================

LoadError: failed process: Process(`curl -f -o c-blosc-1.8.1.tar.gz -L https://github.com/Blosc/c-blosc/archive/v1.8.1.tar.gz`, ProcessExited(77)) [77]
while loading /home/tero/.julia/v0.4/Blosc/deps/build.jl, in expression starting on line 18

================================================================================

=================================================================[ BUILD ERRORS ]==================================================================

WARNING: Blosc had build errors.

 - packages with build errors remain installed in /home/tero/.julia/v0.4
 - build the package(s) and all dependencies with `Pkg.build("Blosc")`
 - build a single package by running its `deps/build.jl` script

===================================================================================================================================================
INFO: Package database updated

julia>

workaround:

bash> cd ~/.julia/v0.4/Blosc/deps/
bash> wget https://github.com/Blosc/c-blosc/archive/v1.8.1.tar.gz
bash> mv v1.8.1.tar.gz c-blosc-1.8.1.tar.gz
bash>

julia> Pkg.build("Blosc")
INFO: Building Blosc

update to blosc 1.5.3

We are currently using 1.5.0, but upstream has had several updates.

Note that the release notes include updates to the blosc HDF5 plugin, so the corresponding changes should be made to HDF5.jl (updating JuliaIO/HDF5.jl#174).

upgrade to c-blosc 1.12.1?

https://github.com/Blosc/c-blosc/releases
Changelog is here: https://github.com/Blosc/c-blosc/blob/master/RELEASE_NOTES.rst

(I need this patch :p Blosc/c-blosc#188

provide additional compressors besides blosclz

The default build of the Blosc package only came with the blosclz compressor. I checked the Blosc build instructions and followed the instructions to build libblosc with internal compressors. Updated build.jl file is in this gist:
https://gist.github.com/robertfeldt/a5433400bc0bdea3996c
(I did not have time to do a proper pull request, sorry). After installing with that build file I now have all 5 Blosc compressors:

julia> Blosc.compressors()
5-element Array{SubString{ASCIIString},1}:
  "blosclz"
  "lz4"
  "lz4hc"
  "snappy"
  "zlib"

and can use them all:

julia> using Base.Test
julia> for (comp, name, _) in Blosc.compressors_info()
       println("Compressor = $comp")
       Blosc.set_compressor(comp)
       for level=0:9
           for shuffle in (true, false)
               for i=1:2048
                   a = rand(Uint8, i)
                   ac = Blosc.compress(a, level=level, shuffle=shuffle)
                   info = Blosc.compressor_info(ac)
                   @test info.library == name
                   @test info.typesize == sizeof(Uint8)
                   @test info.shuffled == shuffle
                   @test Blosc.decompress(Uint8, ac) == a
               end
           end
       end
   end
Compressor = blosclz
Compressor = lz4
Compressor = lz4hc
Compressor = snappy
Compressor = zlib

Blosc.jl does not expose c-blosc's bitshuffle support

The underlying c-blosc library supports three different shuffling modes: no shuffle, byte shuffle, and bit shuffle, but Blosc.jl currently treats shuffling as a boolean choice between no shuffle and byte shuffle. Blosc.jl provides no way to utilize the built-in bit shuffle support of c-blosc.

Download and extraction not working

┌ Error: Error building `Blosc`:
│ [ Info: Downloading https://github.com/Blosc/c-blosc/archive/v1.14.3.tar.gz to C:\Users\Mus\.julia\packages\Blosc\lzFr0\deps\usr\downloads\src.tar.gz...
│ ERROR: LoadError: Could not unpack C:\Users\Mus\.julia\packages\Blosc\lzFr0\deps\usr\downloads\src.tar.gz into C:\Users\Mus\.julia\packages\Blosc\lzFr0\deps\usr\downloads
│ Stacktrace:
│  [1] error(::String) at .\error.jl:33
│  [2] #unpack#98(::Bool, ::Function, ::String, ::String) at C:\Users\Mus\.julia\packages\BinaryProvider\TcAwt\src\PlatformEngines.jl:660
│  [3] #unpack at .\none:0 [inlined]
│  [4] #compile#3(::Prefix, ::Bool, ::Function, ::String, ::String, ::String) at C:\Users\Mus\.julia\packages\Blosc\lzFr0\deps\compile.jl:16
│  [5] (::getfield(Main, Symbol("#kw##compile")))(::NamedTuple{(:prefix, :verbose),Tuple{Prefix,Bool}}, ::typeof(compile), ::String, ::String, ::String) at .\none:0
│  [6] top-level scope at C:\Users\Mus\.julia\packages\Blosc\lzFr0\deps\build.jl:60
│  [7] include at .\boot.jl:326 [inlined]
│  [8] include_relative(::Module, ::String) at .\loading.jl:1038
│  [9] include(::Module, ::String) at .\sysimg.jl:29
│  [10] include(::String) at .\client.jl:403
│  [11] top-level scope at none:0
│ in expression starting at C:\Users\Mus\.julia\packages\Blosc\lzFr0\deps\build.jl:53
│
│ ERROR: The system cannot find the file specified.
│ C:\Users\Mus\.julia\packages\Blosc\lzFr0\deps\usr\downloads\src.tar.gz
│
│
│
│ System ERROR:
│ The system cannot find the file specified.
│
│ 7-Zip 18.05 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2018-04-30
│
│
│ Extracting archive:
│ --
│ Path =
│ Type = tar
│ Code Page = UTF-8
│
│ Everything is Ok
│
│ Files: 0
│ Size:       0
│ Compressed: 0
└ @ Pkg.Operations C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Pkg\src\Operations.jl:1075
  Building HDF5 ─→ `C:\Users\Mus\.julia\packages\HDF5\Y9Znv\deps\build.log`

For some reason the archive never gets downloaded to C:\Users\Mus\.julia\packages\Blosc\lzFr0\deps\usr\downloads

Use new Artifacts infrastructure

A Build Error on linux

Hello, I get a build error on archlinux:

INFO: Building Blosc
===============================================================================[ ERROR: Blosc ]================================================================================

LoadError: could not spawn `make -f ../../make.blosc LIB=../../libblosc.so`: no such file or directory (ENOENT)
while loading /home/smldis/.julia/v0.6/Blosc/deps/build.jl, in expression starting on line 21

there is no makefile on the directory, so build.jl should not call make right?

ls .julia/v0.6/Blosc/deps/c-blosc-1.12.1/ANNOUNCE.rst
 -- files --
ANNOUNCE.rst              blosc/                    cmake_uninstall.cmake.in  README_HEADER.rst         RELEASING.rst             THOUGHTS_FOR_2.0.txt    
appveyor/                 blosc.pc.in               examples/                 README.md                 scripts/                                          
appveyor.yml              cmake/                    internal-complibs/        README_THREADED.rst       tests/                                            
bench/                    CMakeLists.txt            LICENSES/                 RELEASE_NOTES.rst         THANKS.rst

Please, note that I have already that library provided by my system packages. Is there an option to use it?

build failed on ubuntu 18.04

julia> versioninfo()
Julia Version 1.4.0-rc1.0
Commit b0c33b0cf5 (2020-01-23 17:23 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Xeon(R) W-2145 CPU @ 3.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, skylake)

(@v1.4) pkg> build Blosc
    Building CMake → `~/.julia/packages/CMake/ULbyn/deps/build.log`
   Building Blosc → `~/.julia/packages/Blosc/ZgcAM/deps/build.log`
┌ Error: Error building `Blosc`:
│ ┌ Warning: platform_key() is deprecated, use platform_key_abi() from now on
│ │   caller = ip:0x0
│ └ @ Core :-1
│ ERROR: LoadError: LibraryProduct(nothing, ["libblosc"], :libblosc, "Prefix(/home/kong/.julia/packages/Blosc/ZgcAM/deps/usr)") is not satisfied, cannot generate deps.jl!
│ Stacktrace:
│  [1] error(::String) at ./error.jl:33
│  [2] write_deps_file(::String, ::Array{LibraryProduct,1}; verbose::Bool, isolate::Bool) at /home/kong/.julia/packages/BinaryProvider/GeAtj/src/Products.jl:419
│  [3] top-level scope at /home/kong/.julia/packages/Blosc/ZgcAM/deps/build.jl:68
│  [4] include(::String) at ./client.jl:439
│  [5] top-level scope at none:5
│ in expression starting at /home/kong/.julia/packages/Blosc/ZgcAM/deps/build.jl:68
│ LibraryProduct[LibraryProduct(nothing, ["libblosc"], :libblosc, "Prefix(/home/kong/.julia/packages/Blosc/ZgcAM/deps/usr)")]
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/Operations.jl:892

Pkg.build("Blosc") fails

Any idea why this .build("Blosc") fails? This is on fresh install of Bash on Ubuntu 16.04 with Julia v0.5:

julia> Pkg.update()
INFO: Updating METADATA...
INFO: Updating cache of DataFrames...
INFO: Updating cache of DataFrames...
INFO: Computing changes...
INFO: Installing Blosc v0.1.7
INFO: Building Blosc
Compiling libblosc...
================================[ ERROR: Blosc ]================================

LoadError: failed process: Process(make -f ../../make.blosc LIB=../../libblosc.so, ProcessExited(127)) [127]
while loading /root/.julia/v0.5/Blosc/deps/build.jl, in expression starting on line 21

================================================================================

================================[ BUILD ERRORS ]================================

WARNING: Blosc had build errors.

packages with build errors remain installed in /root/.julia/v0.5
build the package(s) and all dependencies with Pkg.build("Blosc")
build a single package by running its deps/build.jl script

================================================================================

julia>

Is the list available compressors mentioned in the README still true?

The README states

Currently, only the blosclz, lz4, and lz4hc algorithms are included

But Blosc.compressors() return

6-element Array{SubString{String},1}:
 "blosclz"
 "lz4"
 "lz4hc"
 "snappy"
 "zlib"
 "zstd"

Segfault after using workspace()

I get a segfault with the following commands in the REPL:

using Blosc
workspace()
using Blosc
exit()

The full error output can be seen here: https://gist.github.com/micahjsmith/60b4556d67c47665ba587b11afa1d6b7

error of building

I am trying to install HDF5 and get the building error, I think that the downloading command do not works for me.
I am using Ubuntu 14.04, julia 4.2, on Google cloud.

INFO: Building Blosc % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (77) error setting certificate verify locations: CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: none ================================[ ERROR: Blosc ]================================ LoadError: failed process: Process(curl -f -o c-blosc-1.7.0.tar.gz -L https://github.com/Blosc/c-blosc/archive/v1.7.0.tar.gz`, ProcessExited(77)) [77]
while loading /usr/local/share/julia/site/v0.4/Blosc/deps/build.jl, in expression starting on line 18
================================================================================`

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Build error on debian machine, julia ver 1.2.0

Hi, I'm getting this build error. Any thoughts on how I can build it properly?

Error: Error building `Blosc`: 
│ ┌ Warning: platform_key() is deprecated, use platform_key_abi() from now on
│ │   caller = ip:0x0
│ └ @ Core :-1
│ ERROR: LoadError: LibraryProduct(nothing, ["libblosc"], :libblosc, "Prefix(/home/ahassan/.juliapro/JuliaPro_v1.2.0-2/packages/Blosc/lzFr0/deps/usr)") is not satisfied, cannot generate deps.jl!
│ Stacktrace:
│  [1] error(::String) at ./error.jl:33
│  [2] #write_deps_file#171(::Bool, ::Bool, ::typeof(write_deps_file), ::String, ::Array{LibraryProduct,1}) at /home/buildbot/build-worker/worker/juliapro-release-centos7-0_6/build/tmp_julia/share/julia/stdlib/v1.2/BinaryProvider/src/Products.jl:419
│  [3] (::getfield(BinaryProvider, Symbol("#kw##write_deps_file")))(::NamedTuple{(:verbose,),Tuple{Bool}}, ::typeof(write_deps_file), ::String, ::Array{LibraryProduct,1}) at ./tuple.jl:0
│  [4] top-level scope at /home/ahassan/.juliapro/JuliaPro_v1.2.0-2/packages/Blosc/lzFr0/deps/build.jl:65
│  [5] include at ./boot.jl:328 [inlined]
│  [6] include_relative(::Module, ::String) at ./loading.jl:1094
│  [7] include(::Module, ::String) at ./Base.jl:31
│  [8] include(::String) at ./client.jl:431
│  [9] top-level scope at none:5
│ in expression starting at /home/ahassan/.juliapro/JuliaPro_v1.2.0-2/packages/Blosc/lzFr0/deps/build.jl:65
└ @ Pkg.Operations /home/buildbot/build-worker/worker/juliapro-release-centos7-0_6/build/tmp_julia/share/julia/stdlib/v1.2/Pkg/src/Operations.jl:1138

Building issues Win64

I am having trouble adding package Blosc.jl on my 64bit Window7 system. Once the process gets to INFO: Building Blosc, nothing seems to happen except the spinning indicator. This used to work fine. I have recently reinstalled Julia and need to add back some of the packages. I don't know if I have made other changes that could have impacted this package. I ran across this problem trying to use MAT.jl. Can you provide some advice on how to proceed?
Thank you.

Test of compressors did not set_compressor

Note that there is a "bug" in the testing of multiple compressors. Fix:

feldt:~/.julia/v0.3/Blosc$ git diff test/runtests.jl
diff --git a/test/runtests.jl b/test/runtests.jl
index bcf00db..5a25212 100644
--- a/test/runtests.jl
+++ b/test/runtests.jl
@@ -47,6 +47,8 @@ ac = Blosc.compress(a)

 # test all compressors
 for (comp, name, _) in Blosc.compressors_info()
+    println("Compressor = $comp")
+    Blosc.set_compressor(comp)
 for level=0:9
     for shuffle in (true, false)
         for i=1:2048

32-bit Windows test failure

See the Appveyor test, and the discussion in JuliaIO/HDF5.jl#179 ; cc @tkelman

The 64-bit Windows c-blosc binary was built with x86_64-w64-mingw32-gcc (GCC) 4.6.3 using x86_64-w64-mingw32-gcc -shared -O3 -msse2 -I. *.c -o libblosc64-1.5.0.dll. And the 32-bit binary was built with i686-w64-mingw32-gcc (GCC) 4.6.3 using i686-w64-mingw32-gcc -shared -O3 -msse2 -I. *.c -o libblosc32-1.5.0.dll.

Build error Julia 0.4.5 on Linux Mint

Pkg.build("Blosc") fails with following message:

Compiling libblosc...
gcc -I. -DSHUFFLE_SSE2_ENABLED -DBLOSC_SHARED_LIBRARY -fPIC -O3 -pthread -msse2 -c -o bitshuffle-sse2.o bitshuffle-sse2.c
In file included from shuffle-common.h:28:0,
                 from bitshuffle-generic.h:18,
                 from bitshuffle-sse2.c:15:
/usr/lib/gcc/x86_64-linux-gnu/4.8/include/stdint.h:9:26: fatal error: stdint.h: No such file or directory
 # include_next <stdint.h>
                          ^
compilation terminated.
make: *** [bitshuffle-sse2.o] Error 1
================================[ ERROR: Blosc ]================================

LoadError: failed process: Process(`make -f ../../make.blosc HAVE_AVX=0 LIB=../../libblosc.so`, ProcessExited(2)) [2]
while loading /home/abieler/.julia/v0.4/Blosc/deps/build.jl, in expression starting on line 21

================================================================================

a quick check in the terminal for stdint.h

locate stdint.h
/usr/lib/gcc/x86_64-linux-gnu/4.8/include/stdint.h
/usr/lib/syslinux/com32/include/stdint.h
/usr/lib/syslinux/com32/include/bitsize/stdint.h
/usr/src/virtualbox-guest-5.0.4/include/iprt/stdint.h
/var/lib/dkms/virtualbox-guest/5.0.4/build/include/iprt/stdint.h

Blosc: v0.3.0 => v0.4.1 failed at CentOS

During the update the package building failed witht the error:

============================================[ ERROR: Blosc ]=============================================

LoadError: BinaryProvider.LibraryProduct("${HOME}/.julia/v0.6/Blosc/deps/usr/lib", String["libblosc"], :libblosc, "Prefix(${HOME}/.julia/v0.6/Blosc/deps/usr)") is not satisfied, cannot generate deps.jl!
while loading ${HOME}/.julia/v0.6/Blosc/deps/build.jl, in expression starting on line 40

=========================================================================================================

It is not clear to me what the problem is.

Previous version still works.

julia> Pkg.pin("Blosc", v"0.3.0")
INFO: Creating Blosc branch pinned.0fdded9b.tmp
INFO: No packages to install, update or remove
julia> Pkg.build("Blosc")
INFO: Building Blosc
julia>

M1 Monterey: libblosc not defined

On Julia 1.7, on an M1, Monterey, I cannot use Blosc anymore. I ran into this problem trying to install HDF5 which had a similar error message but in that case I could simply point it to the home-brew installation of libhdf5. For Blosc I found no instruction how to do this.

Here are my system information and errors:

julia> versioninfo()
Julia Version 1.7.0-rc3.6
Commit 5dba09613d (2021-11-16 11:15 UTC)
Platform Info:
  OS: macOS (arm64-apple-darwin21.1.0)
  CPU: Apple M1 Max
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-12.0.1 (ORCJIT, cyclone)

(@v1.7) pkg> status
      Status `~/.julia/environments/v1.7/Project.toml`
  [a74b3585] Blosc v0.7.1
  [7073ff75] IJulia v1.23.2

After restarting Julia:

   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.7.0-rc3.6 (2021-11-16)
 _/ |\__'_|_|_|\__'_|  |  backports-release-1.7/5dba09613d (fork: 398 commits, 168 days)
|__/                   |

(@v1.7) pkg> build Blosc

julia> using Blosc
ERROR: InitError: UndefVarError: libblosc not defined
Stacktrace:
 [1] __init__()
   @ Blosc ~/.julia/packages/Blosc/vjmKP/src/Blosc.jl:6
 [2] _include_from_serialized(path::String, depmods::Vector{Any})
   @ Base ./loading.jl:768
 [3] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String)
   @ Base ./loading.jl:854
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1097
 [5] require(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1013
 [6] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:997
during initialization of module Blosc

Pkg.test("Blosc") fails- Windows 10, Julia Release 1.0.0

I am trying to install HDF5, but Blosc is failing:

julia> Pkg.test("Blosc")
   Testing Blosc
    Status `C:\Users\vavasis\AppData\Local\Temp\jl_6E15.tmp\Manifest.toml`
  [9e28174c] BinDeps v0.8.9
  [b99e7846] BinaryProvider v0.4.0
  [a74b3585] Blosc v0.5.0
  [d5fb7624] CMakeWrapper v0.2.0
  [34da2185] Compat v1.0.1
  [864edb3b] DataStructures v0.11.0
  [d96e819e] Parameters v0.9.2
  [30578b45] URIParser v0.4.0
  [2a0f44e3] Base64  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\Base64`]
  [ade2ca70] Dates  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\Dates`]
  [8bb1440f] DelimitedFiles  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\DelimitedFiles`]
  [8ba89e20] Distributed  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\Distributed`]
  [b77e0a4c] InteractiveUtils  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\InteractiveUtils`]
  [76f85450] LibGit2  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\LibGit2`]
  [8f399da3] Libdl  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\Libdl`]
  [37e2e46d] LinearAlgebra  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\LinearAlgebra`]
  [56ddb016] Logging  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\Logging`]
  [d6f4376e] Markdown  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\Markdown`]
  [a63ad114] Mmap  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\Mmap`]
  [44cfe95a] Pkg  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\Pkg`]
  [de0858da] Printf  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\Printf`]
  [3fa0cd96] REPL  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\REPL`]
  [9a3f8284] Random  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\Random`]
  [ea8e919c] SHA  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\SHA`]
  [9e88b42a] Serialization  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\Serialization`]
  [1a1011a3] SharedArrays  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\SharedArrays`]
  [6462fe0b] Sockets  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\Sockets`]
  [2f01184e] SparseArrays  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\SparseArrays`]
  [10745b16] Statistics  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\Statistics`]
  [8dfed614] Test  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\Test`]
  [cf7118a7] UUIDs  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\UUIDs`]
  [4ec0a83e] Unicode  [`C:\Users\vavasis\AppData\Local\Julia-1.0.0\bin\..\share\julia\stdlib\v1.0\Unicode`]
ERROR: LoadError: InitError: UndefVarError: Libdl not defined
Stacktrace:
 [1] check_deps() at C:\Users\vavasis\.julia\packages\Blosc\jsnG\deps\deps.jl:13
 [2] __init__() at C:\Users\vavasis\.julia\packages\Blosc\jsnG\src\Blosc.jl:15
 [3] _include_from_serialized(::String, ::Array{Any,1}) at .\loading.jl:627
 [4] macro expansion at .\logging.jl:312 [inlined]
 [5] _require_search_from_serialized(::Base.PkgId, ::String) at .\loading.jl:698
 [6] _require(::Base.PkgId) at .\loading.jl:931
 [7] require(::Base.PkgId) at .\loading.jl:852
 [8] macro expansion at .\logging.jl:311 [inlined]
 [9] require(::Module, ::Symbol) at .\loading.jl:834
 [10] include at .\boot.jl:317 [inlined]
 [11] include_relative(::Module, ::String) at .\loading.jl:1038
 [12] include(::Module, ::String) at .\sysimg.jl:29
 [13] include(::String) at .\client.jl:388
 [14] top-level scope at none:0
during initialization of module Blosc
in expression starting at C:\Users\vavasis\.julia\packages\Blosc\jsnG\test\runtests.jl:1
ERROR: Package Blosc errored during testing
Stacktrace:
 [1] pkgerror(::String, ::Vararg{String,N} where N) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Pkg\src\Types.jl:120
 [2] macro expansion at .\logging.jl:313 [inlined]
 [3] #test#61(::Bool, ::Function, ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Pkg\src\Operations.jl:1283
 [4] #test at .\none:0 [inlined]
 [5] #test#44(::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Pkg\src\API.jl:256
 [6] test at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Pkg\src\API.jl:242 [inlined]
 [7] #test#43 at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Pkg\src\API.jl:239 [inlined]
 [8] test at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Pkg\src\API.jl:239 [inlined]
 [9] #test#42 at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Pkg\src\API.jl:238 [inlined]
 [10] test at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Pkg\src\API.jl:238 [inlined]
 [11] #test#41 at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Pkg\src\API.jl:237 [inlined]
 [12] test(::String) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.0\Pkg\src\API.jl:237
 [13] top-level scope at none:0

julia>