Comments (7)
omad
commented on June 1, 2024
GDAL does support the CF conventions used by storage units for interpreting geospatial coordinates already.
It appears the bounds may be referencing pixel centres instead of edges which will require further investigation.
A simple gdalinfo storageunit.nc lists the available data within the file, and a further call to gdalinfo is required to access the detailed position data. e.g.:
$ gdalinfo LANDSAT_5_TM_148.0_-33.0_NBAR_1996-01-30T22-54-41.500000.nc
Driver: netCDF/Network Common Data Format
Files: LANDSAT_5_TM_148.0_-33.0_NBAR_1996-01-30T22-54-41.500000.nc
Size is 512, 512
...
$ gdalinfo NETCDF:"LANDSAT_5_TM_148.0_-33.0_NBAR_1996-01-30T22-54-41.500000.nc":band_10
Driver: netCDF/Network Common Data Format
Files: LANDSAT_5_TM_148.0_-33.0_NBAR_1996-01-30T22-54-41.500000.nc
Size is 4000, 4000
Coordinate System is:
GEOGCS["unknown",
DATUM["unknown",
SPHEROID["Spheroid",6378137,298.257223563]],
PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433,
AUTHORITY["EPSG","9122"]]]
Origin = (147.999875000000003,-32.999875000000003)
Pixel Size = (0.000250000000000,-0.000250000000000)
Metadata:
band_10#grid_mapping=latitude_longitude
band_10#_FillValue=-999
latitude#axis=Y
latitude#long_name=latitude
latitude#standard_name=latitude
latitude#units=degrees_north
latitude_longitude#grid_mapping_name=latitude_longitude
latitude_longitude#inverse_flattening=298.257223563
latitude_longitude#longitude_of_prime_meridian=0
latitude_longitude#long_name=WGS 84
latitude_longitude#semi_major_axis=6378137
longitude#axis=X
longitude#long_name=longitude
longitude#standard_name=longitude
longitude#units=degrees_east
NC_GLOBAL#Conventions=CF-1.6, ACDD-1.3
NC_GLOBAL#date_created=2015-12-22T09:53:41.438571
NC_GLOBAL#geospatial_bounds=POLYGON((148.0 -33.0, 148.0 -34.0, 149.0 -34.0, 149.0 -33.0, 148.0 -33.0))
NC_GLOBAL#geospatial_bounds_crs=EPSG:4326
NC_GLOBAL#geospatial_lat_max=-33
NC_GLOBAL#geospatial_lat_min=-34
NC_GLOBAL#geospatial_lat_resolution=0.00025 degrees
NC_GLOBAL#geospatial_lat_units=degrees_north
NC_GLOBAL#geospatial_lon_max=149
NC_GLOBAL#geospatial_lon_min=148
NC_GLOBAL#geospatial_lon_resolution=0.00025 degrees
NC_GLOBAL#geospatial_lon_units=degrees_east
NC_GLOBAL#history=NetCDF-CF file created by agdc-v2 at 20151221.
NETCDF_DIM_EXTRA={time}
NETCDF_DIM_time_DEF={1,6}
NETCDF_DIM_time_VALUES=823042481.5
time#axis=T
time#calendar=standard
time#long_name=Time, unix time-stamp
time#standard_name=time
time#units=seconds since 1970-01-01 00:00:00
Corner Coordinates:
Upper Left ( 147.9998750, -32.9998750) (147d59'59.55"E, 32d59'59.55"S)
Lower Left ( 147.9998750, -33.9998750) (147d59'59.55"E, 33d59'59.55"S)
Upper Right ( 148.9998750, -32.9998750) (148d59'59.55"E, 32d59'59.55"S)
Lower Right ( 148.9998750, -33.9998750) (148d59'59.55"E, 33d59'59.55"S)
Center ( 148.4998750, -33.4998750) (148d29'59.55"E, 33d29'59.55"S)
Band 1 Block=500x500 Type=Int16, ColorInterp=Undefined
NoData Value=-999
Metadata:
grid_mapping=latitude_longitude
NETCDF_DIM_time=823042481.5
NETCDF_VARNAME=band_10
_FillValue=-999
from datacube-core.
v0lat1le
commented on June 1, 2024
Will also need to verify that the coordinate system is correctly decoded. osr.SpatialReference.IsSame might be useful for that.
same for Albers...
from datacube-core.
v0lat1le
commented on June 1, 2024
gdal sets 'GetTransform', 'spatial_ref' and possibly 'proj4' attributes on grid_mapping variable corresponding to transform array, crs in WKT format and crs in proj4 format.
might still be worth putting those in after we make sure CF way is working correctly... not sure
from datacube-core.
omad
commented on June 1, 2024
For a basic EPSG:4326 NetCDF4 file created with GDAL from an old geotiff tile, GDAL is doing something strange with the lat/lon variables to get the bounds to look right.
nco.variables['lat'][:]
Out[8]:
array([-10.999875, -10.999625, -10.999375, ..., -10.000625, -10.000375,
-10.000125])
from datacube-core.
sixy6e
commented on June 1, 2024
If by weird you mean inverted, it has probably got to do with the WRITE_BOTTOM_UP creation option.
http://www.gdal.org/frmt_netcdf.html
2 files written by gdal, the second using:
-co "WRITE_BOTTOMUP=NO"
In [7]: lat1[0:10]
Out[7]:
array([-34.999875, -34.999625, -34.999375, -34.999125, -34.998875,
-34.998625, -34.998375, -34.998125, -34.997875, -34.997625])
In [8]: lat2[0:10]
Out[8]:
array([-34.000125, -34.000375, -34.000625, -34.000875, -34.001125,
-34.001375, -34.001625, -34.001875, -34.002125, -34.002375])Not sure if that is related to what you're looking at though.
from datacube-core.
omad
commented on June 1, 2024
Okay, WRITE_BOTTOM_UP is half the weirdness. Both data values and lat values are reversed.
The other half is that the lat/lon values represent pixel centres instead of bounds. Perhaps this is what our NetCDF files should be doing too. It's recommended but not required by the CF conventions.
There was a gdal-dev email thread in 2009 recommending not relying on the meaning of coordinates being centres or otherwise, and to writing code expecting your specific use case.
from datacube-core.
omad
commented on June 1, 2024
I've updated the datacube.model.TileSpec to calculate lat/lon values as pixel centres instead of upper lefts, this matches what GDAL expects and is also what the CF specs recommend (but don't require).
GDAL now recognises the CRS and correctly shows the file boundaries.
For more information the following are interesting email threads:
- CF-Metadata X and Y projection coordinate (center vs corner)
- gdal-dev NetCDF geotransform - pixel centre/upper-left issue.
$ gdalinfo NETCDF:"LANDSAT_5_TM_148_-34_NBAR_1996-01-30T22-54-29.000000.nc":band_10
Driver: netCDF/Network Common Data Format
Files: LANDSAT_5_TM_148_-34_NBAR_1996-01-30T22-54-29.000000.nc
Size is 4000, 4000
Coordinate System is:
GEOGCS["WGS 84",
DATUM["WGS_1984",
SPHEROID["WGS 84",6378137,298.257223563,
AUTHORITY["EPSG","7030"]],
AUTHORITY["EPSG","6326"]],
PRIMEM["Greenwich",0,
AUTHORITY["EPSG","8901"]],
UNIT["degree",0.0174532925199433,
AUTHORITY["EPSG","9122"]],
AUTHORITY["EPSG","4326"]]
Origin = (148.000000000000000,-33.000000000000000)
Pixel Size = (0.000250000000000,-0.000250000000000)
Metadata:
band_10#grid_mapping=latitude_longitude
band_10#_FillValue=-999
latitude#axis=Y
latitude#long_name=latitude
latitude#standard_name=latitude
latitude#units=degrees_north
latitude_longitude#GeoTransform=148 0.00025 0 -33 0 -0.00025
latitude_longitude#grid_mapping_name=latitude_longitude
latitude_longitude#inverse_flattening=298.257223563
latitude_longitude#longitude_of_prime_meridian=0
latitude_longitude#long_name=WGS 84
latitude_longitude#semi_major_axis=6378137
latitude_longitude#spatial_ref=GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4326"]]
longitude#axis=X
longitude#long_name=longitude
longitude#standard_name=longitude
longitude#units=degrees_east
NC_GLOBAL#Conventions=CF-1.6, ACDD-1.3
NC_GLOBAL#date_created=2015-12-23T12:56:23.805598
NC_GLOBAL#geospatial_bounds=POLYGON((148.0 -33.0, 148.0 -34.0, 149.0 -34.0, 149.0 -33.0, 148.0 -33.0))
NC_GLOBAL#geospatial_bounds_crs=EPSG:4326
NC_GLOBAL#geospatial_lat_max=-33
NC_GLOBAL#geospatial_lat_min=-34
NC_GLOBAL#geospatial_lat_resolution=0.00025 degrees
NC_GLOBAL#geospatial_lat_units=degrees_north
NC_GLOBAL#geospatial_lon_max=149
NC_GLOBAL#geospatial_lon_min=148
NC_GLOBAL#geospatial_lon_resolution=0.00025 degrees
NC_GLOBAL#geospatial_lon_units=degrees_east
NC_GLOBAL#history=NetCDF-CF file created by agdc-v2 at 20151223.
NETCDF_DIM_EXTRA={time}
NETCDF_DIM_time_DEF={1,6}
NETCDF_DIM_time_VALUES=823042481.5
time#axis=T
time#calendar=standard
time#long_name=Time, unix time-stamp
time#standard_name=time
time#units=seconds since 1970-01-01 00:00:00
Corner Coordinates:
Upper Left ( 148.0000000, -33.0000000) (148d 0' 0.00"E, 33d 0' 0.00"S)
Lower Left ( 148.0000000, -34.0000000) (148d 0' 0.00"E, 34d 0' 0.00"S)
Upper Right ( 149.0000000, -33.0000000) (149d 0' 0.00"E, 33d 0' 0.00"S)
Lower Right ( 149.0000000, -34.0000000) (149d 0' 0.00"E, 34d 0' 0.00"S)
Center ( 148.5000000, -33.5000000) (148d30' 0.00"E, 33d30' 0.00"S)
Band 1 Block=500x500 Type=Int16, ColorInterp=Undefined
NoData Value=-999
Metadata:
grid_mapping=latitude_longitude
NETCDF_DIM_time=823042481.5
NETCDF_VARNAME=band_10
_FillValue=-999
from datacube-core.
Related Issues (20)
- Deprecation Warnings from `jsonschema` and SQLAlchemy HOT 1
- Dask load returning all nan values HOT 3
- Support for per-grid CRS, for better STAC interoperability
- API Query doctest fails due to different timezone representation
- --with-docker unable to connect to local postgres when running integration tests HOT 1
- Errors following documentation build instructions HOT 1
- AttributeError: module 'ee.data' has no attribute '_get_cloud_api_resource' HOT 1
- issue in find_less_mature when indexing datasets without region_code HOT 3
- Installation environment issues HOT 2
- Error running `datacube system init --no-init-users` HOT 1
- datacube.utils.documents.UnknownMetadataType: Unknown metadata type: 'eo3' while running the command datacube product add s2_l2a.odc-product.yaml in opendatacube HOT 4
- Record intentional omissions from collections HOT 3
- Compatibility issue - python version and aiohttp HOT 6
- Unable to Work with Temporary AWS Security Credentials HOT 6
- lost date time and measurements after ingesting the data using the template s2amsil1c_albers_10.yaml HOT 3
- ODC feature request: env var for `skip_broken_datasets`? HOT 2
- list_products ignores information in load hints
- Replace datacube GeoBox with odc-geo GeoBox? HOT 1
- Use odc-geo GridSpec model internally HOT 6
- Cannot pass `odc.geo` GeoBoxes to `dc.load` HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from datacube-core.