Comments (8)
What error do you get in the stack ? There may be a baked in --no-sign-request
somewhere. Have you tried to use OWS as a library in the same environment to render some layers and seen the results ?
Could you try "false" as the value for no-sign request and see the results. Reading the tests, looks like this is what is tested. We should improve the docs to clearly show the valid string values interpreted as true and false.
from datacube-ows.
If I setLevel
of logging to INFO
, I can see that:
[2023-07-24 16:46:50,542] [INFO] S3 access configured with signed requests
[2023-07-24 16:46:50,543] [INFO] Establishing/renewing credentials
[2023-07-24 16:46:50,634] [INFO] Found credentials in environment variables.
indicating that there's no issue regarding absence of these env vars.
I suppose datacube-ows
queries datacube
, which returns xarrays. If this assumption holds, I could look into what's being returned and debug from there. What do you think?
Btw, changing to 'false' didn't have any effect...
from datacube-ows.
All of the read handling using OWS is done by datacube, best to capture some intermediate results there and check. It would be also good to hear from the community using authenticated S3 buckets with OWS.
from datacube-ows.
We're running OWS with a private S3 bucket (disclaimer: we're running the fairly old version 1.8.18
of datacube-ows
still)
In the config, the only S3 related item we have set is s3_aws_zone
(so we don't specify name / url).
In the environment variables we set:
AWS_NO_SIGN_REQUEST=NO
AWS_DEFAULT_REGION=eu-central-1
We don't set credentials as these are supplied by the attached role and acquired / refreshed by the datacube-ows
app.
Can you confirm that
from datacube import Datacube dc = Datacube() dss = dc.find_datasets(product='s2_l2a_10m_v1') # returns the correct xarray Dataset just fine: data = dc.load( datasets=dss, latitude=(55.52, 55.7), longitude=(12.6, 12.75), output_crs="EPSG:3857", resolution=(-100, 100), )
is actually loading the data (it should) and not a dask array? Also, is that run from the same environment as OWS?
from datacube-ows.
I confirm that the dc.load function I included returns a properly populated xarray.Dataset that I can visualize with matplotlib and see the actual raster data from the S3. And, yes, it's within the same env; the reason I included it is because I wanted to show exactly that, i.e., that the datacube
can access my S3 rasters just fine. I'm not sure but it seems the problem lies between the datacube-core and the datacube-ows, if that makes sense...
from datacube-ows.
I remember it can be tricky to sort out loading issues between these layers. If I'd had to guess, either somehow requests are set to unsigned, the credentials are somehow not picked up, or you are using a different set of credentials that don't have the required permissions.
You probably need to dig into the logs a bit, set to debug and so on, to see what's happening behind the scenes with botocore
/ rasterio
etc.
You say that OWS does not deliver raster data. Perhaps you can share an error message or any other hint to why that is the case.
from datacube-ows.
Have you rerun datacube-ows-update --views; datacube-ows-update
?
OWS does not use the same search mechanism as core - it has a separate postgis index to the database that allows more accurate spatial searches, which must be maintained as per the documentation here.
from datacube-ows.
I'm assuming we can close this now.
from datacube-ows.
Related Issues (20)
- XML error results are returned with 200 status code HOT 11
- Error creating New Space Materialised View for sinusoidal(WKT) products in v1.8.x HOT 4
- Feature request: Allow aggregator function to receive raw band data instead of index function output
- Creating extents for freshly indexed products fails with unfriendly error
- WCS rendering to GeoTiff fails if NODATA is out of range HOT 3
- GetCapability generation fails if internal tables are stale HOT 2
- WCS2 is not checking for case where user specifies NONE of scaling/subsetting/etc. HOT 1
- Generation of invalid GetCapabilities responses when layers are not configured in a single root HOT 1
- Running `datacube-ows --version` doesn't output a version HOT 2
- Prometheus Metrics - Noise From Prometheus Probes HOT 1
- The use of datacube-ows-update is poorly documented HOT 11
- New Feature: date display mapping HOT 6
- some dependencies are too out-of-date for python3.10
- docker-compose failing for 1.8.35 HOT 2
- Setting the value of $PYDEV_DEBUG HOT 1
- Pluggable JWT validator to blueprints HOT 2
- Timezone issue in feat info HOT 4
- Feature request: expose dataset metadata fields in Feature Info HOT 4
- Change reference in error/help string
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from datacube-ows.