Discuss best practices for `xr.Dataset.to_netcdf()` about bitinformation.jl HOT 3 OPEN

What I do not want is that the compression impacts the read performance too much, i.e. reading the compressed data should not take much more (subjective I know) time that compared to before (or is this the tradeoff anyways).

Many lossless compressors have a decompression speed that's approximately independent of the compression level. So read performance is rarely something to worry about with this method. E.g.

(from https://www.nature.com/articles/s43588-021-00156-2)

from bitinformation.jl.

Some further interesting reading for me:

• https://www.unidata.ucar.edu/blogs/developer/entry/netcdf_compression
• https://unidata.github.io/netcdf4-python/#efficient-compression-of-netcdf-variables: use unlimited_dims=[] in to_netcdf()
• https://discourse.pangeo.io/t/xarray-and-compression-options-for-large-netcdf-files/1154/8
• video training: https://www.youtube.com/watch?v=ml4RkEaB4DQ

from bitinformation.jl.

compress and save with xarray in xbitinfo: https://xbitinfo.readthedocs.io/en/latest/api.html#save-compressed

from bitinformation.jl.