Comments (5)
I have data along dimensions longitude, latitude and time and somehow intuitively would run the analysis along time.
You can run the analysis along any dimension you like. You can also add the information. The first dimension is usually just the default because that's also how the data is layed out in memory/on disk. Things can change along different dimensions, depending on the resolution. Check the supplement of our paper for some examples.
from bitinformation.jl.
That test is indeed confusing. As the array A
is not sorted, every entry is independent of the next hence all those tests just check that the information is zero.
julia> using BitInformation
julia> A = rand(Float32,30,40,50);
julia> bi1 = bitinformation(A,dim=1);
julia> bi2 = bitinformation(A,dim=2);
julia> bi3 = bitinformation(A,dim=3);
julia> hcat(bi1,bi2,bi3)
32×3 Matrix{Float64}:
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0
⋮
However, if you sort the array in a given dimension then you artificially introduce some information, which is highest in that dimension
julia> sort!(A,dims=1);
julia> bi1 = bitinformation(A,dim=1);
julia> bi2 = bitinformation(A,dim=2);
julia> bi3 = bitinformation(A,dim=3);
julia> hcat(bi1,bi2,bi3)
32×3 Matrix{Float64}:
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0
0.0067747 0.00508132 0.00538892
0.292094 0.182393 0.187531
0.550684 0.265361 0.271625
0.371526 0.114251 0.118072
0.237596 0.0441321 0.0441709
⋮
0.0 0.0 9.3149e-5
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.000280003
0.000749177 0.000946589 0.000850585
0.00515332 0.00430802 0.00508684
0.0233246 0.0177343 0.0185884
0.061388 0.0458432 0.0484664
bi1
will have the highest information in the exponent/mantissa bits, but sorting along 1 dimension also influences the other (with smaller information though). The information in the last mantissa bits is due to the poor sampling of rand
(see the randfloat
function in JuliaRandom/RandomNumbers.jl as an alternative).
from bitinformation.jl.
is it also possible to run bitinformation
on all dimensions and does that make sense?
from bitinformation.jl.
Yes, that's the same as running it in all dimensions separately and averaging the information. As it's an arithmetic mean you'll end up in the situation that if the information is high in one dimension but low in another that you may cut off too many bits for that high-information dimension. So what I often just went for is using longitude alone. Rule of thumb that I found in our data is information is highest in longitude/time then latitude then vertical then ensemble. But that obviously depends on the spatio-temporal resolution...
from bitinformation.jl.
thank you
from bitinformation.jl.
Related Issues (16)
- TagBot trigger issue HOT 10
- @inbounds for array rounding HOT 1
- Where is the best place to discuss usage/interpretation/best practices? HOT 30
- Applying BitInformation to compress WRF model results HOT 4
- Discuss best practices for `xr.Dataset.to_netcdf()` HOT 3
- Bitinformation of masked data HOT 14
- Incorrect round away from zero for keepbits=significand_bits HOT 1
- Understanding latitudinal bounds of bitrounding absolute error HOT 11
- Improve Error message when `dim` in `bitinformation(data, dim)` too short HOT 3
- How to implement boundary conditions with `masked_value` HOT 3
- Method definition triggers warning in precompilation HOT 1
- Compressing zarr data store for simulation data HOT 6
- Smallest chunk based on statistics of random information HOT 1
- BitInformation for data previously reduced in precision HOT 5
- Check for NaNs and raise warning
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bitinformation.jl.