Comments (4)
alexamici
commented on August 15, 2024
1
We don't support having mixed stepUnits for the same variable and you can easily work-around that using the filter_by_keys keyword argument in open_dataset, as documented in the README. In this sense the issue is a duplicate of #2, please move there if you have additional comments. I'll close this one.
For example:
>>> xarray_store.open_dataset(
... 'cosmo-d2_germany_regular-lat-lon_single-level_2018073000_001_ASWDIFD_S.grib2',
... filter_by_keys={'stepUnits': 1})
Out[9]:
<xarray.Dataset>
Dimensions: (latitude: 701, longitude: 851)
Coordinates:
time datetime64[ns] ...
step timedelta64[ns] ...
surface int64 ...
* latitude (latitude) float64 43.0 43.02 43.04 43.06 43.08 43.1 43.12 ...
* longitude (longitude) float64 0.0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 ...
valid_time datetime64[ns] ...
Data variables:
unknown (latitude, longitude) float32 ...
Attributes:
GRIB_edition: 2
GRIB_centre: edzw
GRIB_centreDescription: Offenbach
GRIB_subCentre: 255
history: GRIB to CDM+CF via cfgrib-0.8.4.1/ecCodes-2.7.0
Anyway, your GRIB file is fairly interesting, for a number of reasons, thanks for sharing. In fact now we will hit a second problem, ecCodes returns unknown as a cfVarName, that use as variable name in the NetCDF mapping without checking it. I'll keep track of this problem in another issue.
from cfgrib.
alexamici
commented on August 15, 2024
1
@meteoDaniel can you share the fail that is failing with Wrong message length? If you can please open a new issue and attach it, thanks!
from cfgrib.
meteoDaniel
commented on August 15, 2024
Thanks for your support. I got this idea a few minutes ago, too.
I read an article on linkedin about this software project. Your colleague wrote that you need feedback. So i hope this will help you to improve cfgrib. Could you tell me how i can get the values output for this "unkown" variable?
And with another girb file i got an EcCodesError, maybe you know how to deal with it too:
EcCodesError: ('Wrong message length (-23).', -23)
With grib_get_data by eccodes we have no problem to open the file.
Thanks a lot ;)
from cfgrib.
alexamici
commented on August 15, 2024
Please refer to the xarray documentation to see how to work with xarray.Dataset and xarray.DataArray objects.
Short intro:
>>> ds = open_dataset(...)
>>> ds
<xarray.Dataset>
Dimensions: (latitude: 701, longitude: 851, step: 3)
Coordinates:
time datetime64[ns] ...
* step (step) timedelta64[ns] 01:15:00 01:30:00 01:45:00
surface int64 ...
* latitude (latitude) float64 43.0 43.02 43.04 43.06 43.08 43.1 43.12 ...
* longitude (longitude) float64 0.0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 ...
valid_time (step) datetime64[ns] ...
Data variables:
unknown (step, latitude, longitude) float32 ...
Attributes:
GRIB_edition: 2
GRIB_centre: edzw
GRIB_centreDescription: Offenbach
GRIB_subCentre: 255
history: GRIB to CDM+CF via cfgrib-0.8.4.1/ecCodes-2.7.0
>>> da = ds.unkown
>>> da
<xarray.DataArray 'unknown' (step: 3, latitude: 701, longitude: 851)>
[1789653 values with dtype=float32]
Coordinates:
time datetime64[ns] ...
* step (step) timedelta64[ns] 01:15:00 01:30:00 01:45:00
surface int64 ...
* latitude (latitude) float64 43.0 43.02 43.04 43.06 43.08 43.1 43.12 ...
* longitude (longitude) float64 0.0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 ...
valid_time (step) datetime64[ns] ...
Attributes:
GRIB_paramId: 500481
GRIB_shortName: ASWDIFD_S
GRIB_units: W m-2
GRIB_name: Downward diffusive short wave r...
GRIB_cfName: unknown
GRIB_cfVarName: unknown
GRIB_missingValue: 9999
GRIB_totalNumber: undef
GRIB_gridType: regular_ll
GRIB_gridDefinitionDescription: Latitude/longitude
GRIB_numberOfPoints: 596551
GRIB_stepUnits: 0
GRIB_stepType: avg
GRIB_typeOfLevel: surface
GRIB_NV: 0
GRIB_Ni: 851
GRIB_iDirectionIncrementInDegrees: 0.02
GRIB_iScansNegatively: 0
GRIB_longitudeOfFirstGridPointInDegrees: 0.0
GRIB_longitudeOfLastGridPointInDegrees: 17.0
GRIB_Nj: 701
GRIB_jDirectionIncrementInDegrees: 0.02
GRIB_jPointsAreConsecutive: 0
GRIB_jScansPositively: 1
GRIB_latitudeOfFirstGridPointInDegrees: 43.0
GRIB_latitudeOfLastGridPointInDegrees: 57.0
standard_name: unknown
long_name: Downward diffusive short wave r...
units: W m-2
>>> da.values
array([[[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
...,
[ 0., 0., 0., ..., 0., 0., 0.],
[ 0., 0., 0., ..., 0., 0., 0.],
[ 0., 0., 0., ..., 0., 0., 0.]],
[[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
...,
[ 0., 0., 0., ..., 0., 0., 0.],
[ 0., 0., 0., ..., 0., 0., 0.],
[ 0., 0., 0., ..., 0., 0., 0.]],
[[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
...,
[ 0., 0., 0., ..., 0., 0., 0.],
[ 0., 0., 0., ..., 0., 0., 0.],
[ 0., 0., 0., ..., 0., 0., 0.]]], dtype=float32)
Note that the values have been reshaped to a proper lat lon grid.
Please ask other user support question on StackOverflow, not here.
from cfgrib.
Related Issues (20)
- open_datasets fails to open GRIB messages of same parameter with different forecastTime values, silently skipping them HOT 4
- have cfgrib exit on skipping variable HOT 2
- xarray_to_grib.expand_dims too aggressively tries to expand dimensions that are already represented? HOT 4
- Inconsistency between "grib_errors" and "errors" as argument name HOT 1
- Restrictions on reusing an index file are too restrictive HOT 6
- Add to ECMWF downstream ci
- Split cf2cdm out of cfgrib and move it to its own repo HOT 4
- Unable to read the correct time series from NCEP .grib2 HOT 1
- nws noaa grib files open_file but fail to convert OnDiskArray to numpy array (cfgrib >= 0.9.10.2) HOT 4
- Reading National Blend of Models (NBM) Data with cfgrib HOT 6
- broken link in example grib file in github Readme "Usage" section HOT 1
- Offset Longitude Values in FNL GRIB1 files.
- Wrong latitude and longitude HOT 2
- Value error on CfMessage `step` key HOT 3
- Unable to read "step" with ecCodes 2.34.0 HOT 1
- Select dataset/group while opening grib file with xarray
- Request a new release HOT 3
- Cannot find eccodes when using cfgrib HOT 6
- cfgrib and pyinstaller HOT 2
- cfgrib loses all sub-hourly data with newer eccodes version HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cfgrib.