stac-utils / qgis-stac-plugin Goto Github PK
View Code? Open in Web Editor NEWQGIS plugin for reading STAC APIs
License: GNU General Public License v3.0
QGIS plugin for reading STAC APIs
License: GNU General Public License v3.0
Add basic framework on how the main functionality for browsing the STAC APIs will be implemented, this includes defining the models, api client logic and the use of the third party libraries.
Hi,
Thank you for this plugin, it is an important contribution to the infrastructure around STAC.
I have tried the plugin against our STAC API test server and trying to load collections from it will crash QGIS completely. Maybe something is wrong with our service, but the plugin shouldnt crash QGIS - even with a malfunctioning API, I think.
We are not too keen on giving wide access to the service, but we will be happy to share the URL with the plugin team. Just write me at asger @ septima .dk
The search filters values used when searching for items should be stored and loaded when plugin is opened the next time.
Using the plugin settings_manager
, implement the store_search_filters
and get_search_filters
functions then use them in the main plugin widget to save and retrieve the filters.
The design for the result tab will include using a dedicated search result widget for showing search results, the widget will contain details and actions buttons for the result items.
We need to implement and add basic GUI functionality for the widget.
Below is the initial suggested mockup for the widget.
Provide settings and procedure for the plugin to be able to load and save connections details from and to files.
Currently the plugins test running on the Github actions are executed on a linux environment. We need to explore the possibility of running the tests in other OS environments eg.Windows and macOS.
This will enable the plugin tests to be checked in multiple environments in one go.
see https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners
Provide feedback and suggestions on the way forward for this task.
The planetary computer STAC API uses SAS token for access of some assets see https://planetarycomputer.microsoft.com/docs/concepts/sas/.
Add support in the plugin for user to be able to use the authentication api to access assets in the planetary computer API.
Really useful plugin! Thanks!
The View Asset window, once opened, is not able to be closed (there is no button to close the window), which then blocks returning back to the stac browser window or even main map screen. I am forced to kill qgis via the terminal.
OS: Linux Mint 20.2
QGIS: 3.22.3
I'm finding that on my laptop (which has a relatively small screen), the main dialog window opens in such a way that it doesn't fit on the screen properly. The 'search' button at the bottom is off the bottom of the screen, and therefore can't be clicked. I've tried resizing the dialog, but that doesn't seem to work - presumably because there are too many controls to fit in a smaller dialog.
I've got two ideas for how to deal with this:
After search is complete, add the ability to download item assets and loading the item footprint into QGIS.
The current search item result widget provides buttons to load footprint and a combo box for downloading assets, we need to add functionality in the plugin to use the buttons to let user download assets and load the STAC items footprints.
Implement a new dialog that will show collection detail view, with relevant collection attributes
(description, license, etc). Also include a link with rel "describedby" for external details.
Currently the plugin only highlight the collection properties in collection group box, see below screenshot
Plugin collection group box, showing a list of collections from Microsoft planetary computer catalog (https://planetarycomputer.microsoft.com/api/stac/v1)
We need a dedicated dialog that will display more information about a particular collection, see https://github.com/radiantearth/stac-spec/blob/master/collection-spec/collection-spec.md#collection-fields for a list of properties that a collection can contain.
Add a form box in the search tab that will allow using advance queries when searching for items in the STAC API.
The information about filters that can be used is found here https://github.com/radiantearth/stac-api-spec/tree/master/item-search#filter.
The form box tooltip should notify the user that the used filter language will only work if it is supported by the connection STAC API.
STAC API filter fragment https://github.com/radiantearth/stac-api-spec/tree/master/fragments/filter#stac-api---filter-fragment
The returned searched items in the plugin should be paginated to support browsing STAC APIs with large datasets.
See https://github.com/radiantearth/stac-api-spec/blob/master/item-search/examples.md#paging-examples.
We need to change the connection dialog window title to Edit Connection
when opened in a edit mode and when fetching conformance classes we should disable all other window UI components.
Image showing a connection dialog showing Create New Connection
title when opened in edit mode
Screenshot of connection dialog not disabling UI components when fetching
Using the available Add footprint
button, implement support for loading items footprints inside QGIS as map layers.
When searching for the first after installing the plugin an AttributeError happens
An error has occurred while executing Python code:
AttributeError: 'NoneType' object has no attribute 'get_items'
Traceback (most recent call last):
File "/home/samweli/.local/share/QGIS/QGIS3/profiles/stac_install/python/plugins/qgis_stac/gui/qgis_stac_widget.py", line 227, in search_items
self.api_client.get_items(
AttributeError: 'NoneType' object has no attribute 'get_items'
Python version: 3.8.10 (default, Sep 28 2021, 16:10:42) [GCC 9.3.0]
QGIS version: 3.22.0-Białowieża Białowieża, d9022691f1
Python Path:
/usr/share/qgis/python
/home/samweli/.local/share/QGIS/QGIS3/profiles/stac_install/python
/home/samweli/.local/share/QGIS/QGIS3/profiles/stac_install/python/plugins
/usr/share/qgis/python/plugins
/usr/lib/python38.zip
/usr/lib/python3.8
/usr/lib/python3.8/lib-dynload
/home/samweli/.local/lib/python3.8/site-packages
/usr/local/lib/python3.8/dist-packages
/usr/lib/python3/dist-packages
/home/samweli/.local/share/QGIS/QGIS3/profiles/stac_install/python
/home/samweli/.local/share/QGIS/QGIS3/profiles/stac_install/python/plugins/qgis_stac/lib
Firstly: this is a great plugin, thank you. I've wanted a QGIS plugin for STAC for ages, and this is absolutely brilliant! Now on to the actual issue:
The version of the pystac library that is included with this plugin is slightly out of date (understandably), but that means it doesn't include this bugfix. Without that PR, pystac fails to list the collections for any of my STAC API servers.
I'll submit a PR shortly to upgrade the included pystac.
When loading the plugin for the first time, make sure we set the planetary computer as a default connection. The API is found here https://planetarycomputer.microsoft.com/api/stac/v1
Add search implementation for the STAC API items and collections in the plugin.
This includes using the extent, temporal and other provided fields in UI to filter the available STAC Catalogs. see https://github.com/radiantearth/stac-api-spec/tree/master/fragments/filter for filtering docs.
For reference, the items search parameters table https://github.com/radiantearth/stac-api-spec/tree/master/item-search#query-parameter-table
When trying to load COG layers via QgsTask, QGIS application doesn't respond or crashes.
This error is found on draft branch found here https://github.com/Samweli/qgis-stac-plugin/tree/add_layers
Due to recent plugin updates the results window no more uses QTreeView due to crashes that were caused by the delegate used by the view when painting result item widgets, now we are using QScrollArea as main view for the widgets..
QTreeView support items filtering and sorting out of the box using QSortFilterProxyModel but QScrollArea doesn't, so we need to implement these functionality inside the plugin.
Create a plugin message bar that will be responsible for providing notifications about the progress of plugin operations to user.
The progress bar should not distort the UI when it is shown.
Implement logic to handle when edit and remove buttons for connections details have been clicked.
The remove button should prompt user to confirm about the action before removing the connection.
When installing or upgrading the plugin in Windows OS with QGIS 3.16.14 an error occurs indicating there is a missing module called
typing_extensions
Couldn't load plugin 'qgis_stac' due to an error when calling its classFactory() method
ModuleNotFoundError: No module named 'typing_extensions'
Traceback (most recent call last):
File "C:/PROGRA~1/QGIS3~1.16/apps/qgis-ltr/./python\qgis\utils.py", line 334, in _startPlugin
plugins[packageName] = package.classFactory(iface)
File "C:/Users/USER/AppData/Roaming/QGIS/QGIS3\profiles\yu/python/plugins\qgis_stac\__init__.py", line 41, in classFactory
from .main import QgisStac
File "C:/PROGRA~1/QGIS3~1.16/apps/qgis-ltr/./python\qgis\utils.py", line 793, in _import
mod = _builtin_import(name, globals, locals, fromlist, level)
File "C:/Users/USER/AppData/Roaming/QGIS/QGIS3\profiles\yu/python/plugins\qgis_stac\main.py", line 23, in
from .gui.qgis_stac_widget import QgisStacWidget
File "C:/PROGRA~1/QGIS3~1.16/apps/qgis-ltr/./python\qgis\utils.py", line 793, in _import
mod = _builtin_import(name, globals, locals, fromlist, level)
File "C:/Users/USER/AppData/Roaming/QGIS/QGIS3\profiles\yu/python/plugins\qgis_stac\gui\qgis_stac_widget.py", line 21, in
from ..api.client import Client
File "C:/PROGRA~1/QGIS3~1.16/apps/qgis-ltr/./python\qgis\utils.py", line 793, in _import
mod = _builtin_import(name, globals, locals, fromlist, level)
File "C:/Users/USER/AppData/Roaming/QGIS/QGIS3\profiles\yu/python/plugins\qgis_stac\api\client.py", line 3, in
from .base import BaseClient
File "C:/PROGRA~1/QGIS3~1.16/apps/qgis-ltr/./python\qgis\utils.py", line 793, in _import
mod = _builtin_import(name, globals, locals, fromlist, level)
File "C:/Users/USER/AppData/Roaming/QGIS/QGIS3\profiles\yu/python/plugins\qgis_stac\api\base.py", line 19, in
from .network import ContentFetcherTask
File "C:/PROGRA~1/QGIS3~1.16/apps/qgis-ltr/./python\qgis\utils.py", line 793, in _import
mod = _builtin_import(name, globals, locals, fromlist, level)
File "C:/Users/USER/AppData/Roaming/QGIS/QGIS3\profiles\yu/python/plugins\qgis_stac\api\network.py", line 20, in
from pystac_client import Client
File "C:/PROGRA~1/QGIS3~1.16/apps/qgis-ltr/./python\qgis\utils.py", line 793, in _import
mod = _builtin_import(name, globals, locals, fromlist, level)
File "C:\Users\USER\AppData\Roaming\QGIS\QGIS3\profiles\yu\python\plugins\qgis_stac\lib\pystac_client\__init__.py", line 4, in
from pystac_client.item_search import ItemSearch
File "C:/PROGRA~1/QGIS3~1.16/apps/qgis-ltr/./python\qgis\utils.py", line 793, in _import
mod = _builtin_import(name, globals, locals, fromlist, level)
File "C:\Users\USER\AppData\Roaming\QGIS\QGIS3\profiles\yu\python\plugins\qgis_stac\lib\pystac_client\item_search.py", line 11, in
from pystac import Collection, Item, ItemCollection
File "C:/PROGRA~1/QGIS3~1.16/apps/qgis-ltr/./python\qgis\utils.py", line 793, in _import
mod = _builtin_import(name, globals, locals, fromlist, level)
File "C:\Users\USER\AppData\Roaming\QGIS\QGIS3\profiles\yu\python\plugins\qgis_stac\lib\pystac\__init__.py", line 65, in
from pystac.collection import (
File "C:/PROGRA~1/QGIS3~1.16/apps/qgis-ltr/./python\qgis\utils.py", line 793, in _import
mod = _builtin_import(name, globals, locals, fromlist, level)
File "C:\Users\USER\AppData\Roaming\QGIS\QGIS3\profiles\yu\python\plugins\qgis_stac\lib\pystac\collection.py", line 32, in
from pystac.summaries import Summaries
File "C:/PROGRA~1/QGIS3~1.16/apps/qgis-ltr/./python\qgis\utils.py", line 793, in _import
mod = _builtin_import(name, globals, locals, fromlist, level)
File "C:\Users\USER\AppData\Roaming\QGIS\QGIS3\profiles\yu\python\plugins\qgis_stac\lib\pystac\summaries.py", line 24, in
from typing_extensions import Protocol
File "C:/PROGRA~1/QGIS3~1.16/apps/qgis-ltr/./python\qgis\utils.py", line 793, in _import
mod = _builtin_import(name, globals, locals, fromlist, level)
ModuleNotFoundError: No module named 'typing_extensions'
Python version: 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)]
QGIS version: 3.16.14-Hannover Hannover, 4279198632
Python Path:
C:/PROGRA~1/QGIS3~1.16/apps/qgis-ltr/./python
C:/Users/USER/AppData/Roaming/QGIS/QGIS3\profiles\yu/python
C:/Users/USER/AppData/Roaming/QGIS/QGIS3\profiles\yu/python/plugins
C:/PROGRA~1/QGIS3~1.16/apps/qgis-ltr/./python/plugins
C:\PROGRA~1\QGIS3~1.16\apps\Python37
C:\PROGRA~1\QGIS3~1.16\apps\Python37\Scripts
C:\Program Files\QGIS 3.16\bin\python37.zip
C:\PROGRA~1\QGIS3~1.16\apps\Python37\DLLs
C:\PROGRA~1\QGIS3~1.16\apps\Python37\lib
C:\Program Files\QGIS 3.16\bin
C:\PROGRA~1\QGIS3~1.16\apps\Python37\lib\site-packages
C:\PROGRA~1\QGIS3~1.16\apps\Python37\lib\site-packages\win32
C:\PROGRA~1\QGIS3~1.16\apps\Python37\lib\site-packages\win32\lib
C:\PROGRA~1\QGIS3~1.16\apps\Python37\lib\site-packages\Pythonwin
C:/Users/USER/AppData/Roaming/QGIS/QGIS3\profiles\yu/python
C:\Users\USER\AppData\Roaming\QGIS\QGIS3\profiles\yu\python\plugins\qgis_stac\lib
see screenshot of the error below
Initially reported by @mmcfarland
It would be useful to also read static STAC catalogs.
Initially I'd say this would work by checking if the URL is a JSON document, then loading it and disabling the UI portions related to search, filtering, and pagination.
Add a list of open STAC APIs in the plugin as pre-populated connections. One example of the open STAC APIs is https://earth-search.aws.element84.com/v0
When loading thumbnail images into the search result item widgets, thumbnail do not load properly or sometimes QGIS crashes.
Work branch https://github.com/Samweli/qgis-stac-plugin/tree/fix_thumbnail
When adding the item footprint, it appears that the geometry is added as a new layer. The item object itself is a valid GeoJSON object, and has valuable metadata in its properties
field. Is it possible to just use the Item GeoJSON directly, and not create a new geometry so that the user can access the properties through QGIS?
I'm assuming this isn't the case currently, because there were not properties on the added footprints:
We noticed an issue preventing the interoperability with implementations that present different view of the data based on the Accept
header. For example the GeoServer implementation uses the same endpoint for presenting an HTML view but also the STAC representations.
This could be easily fixed by including the Accept: application/geojson
in all HTTP requests.
P.S.
One such endpoint is: https://registry.sage.uvt.ro/geoserver/ogc/stac
When displaying thumbnail for the STAC item, we should use item asset with role overview
for thumbnail image if thumbnail
is not available for the item. See https://github.com/radiantearth/stac-spec/blob/master/item-spec/item-spec.md#asset-roles for the available asset roles.
The asset with role overview
usually contain possibly larger view than the thumbnail of the Item, we need to append height/width params to the URL to get a right size for thumbnail purposes.
Add settings manager in the plugin to handle user, UI and global plugin settings management. The manager should be based on the QgsSettings
https://qgis.org/pyqgis/3.2/core/other/QgsSettings.html.
Add common metadata and extension properties as filters in the search tab.
For each connection, add a tool button to search and show the available conformance classes. This will enable knowing what operations are supported by the STAC API before using the connection in the plugin.
See https://github.com/radiantearth/stac-api-spec/blob/master/overview.md#conformance-classes for more information about conformance
After the item search is complete, we need to load and show the found results into the results tab. The results list should be a list of items in form of search result widgets. Also we need to add ability to filter and sort the results in UI see https://doc.qt.io/qt-5/qsortfilterproxymodel.html, implement QgsTasks for handling fetching, displaying an item thumbnail and adding items assets and footprints as layers inside QGIS.
Rolling list of UI issues, suggestions, or considerations.
title
attribute. When they don't, the plugin list them in the asset dropdown as an empty string, which looks broken (or is at least unhelpful to the user). To account for this, the dropdown should use the asset Dict key as a suitable alternative to the title
(e.g., SR_B1
). An example can be found when viewing items on the Esri 10m Land Cover dataset.Add scripts to enable running plugin tests locally on Windows and macOS operating systems.
The scripts should use the same approach used in the plugins linux bash script here https://github.com/stac-utils/qgis-stac-plugin/blob/main/run-tests.sh, which uses docker images, a test_suite module and QGIS testing environment to run the plugin tests.
After adding the scripts, add a section on the README file to explain how to use them.
HI!
This seems to be a perfect plugin for QGIS users in Sweden. The Swedish NLS are shifting their access to API in favor of downloads and wms.
I have a working QGIS API connection to the Swedish NLS geodata storage. I have all credentials and can access the API as a standard WFS/OGC API connection within QGIS. However, trying the same access information and my existing OAuth2 auth configuration in the qgis-stac-plugin, I get the errors below. Where can I put in the header they are asking for?
NOTE: The sample API connections provided with the plugin are working, and I can add assets and stuff. The problem seems to be in is in the authorisation part.
2022-02-12T08:59:51 INFO Testing connection...
2022-02-12T08:59:51 CRITICAL Connection is not a valid STAC API
2022-02-12T08:59:59 INFO Getting API conformance classes...
2022-02-12T08:59:59 WARNING Problem in fetching content for https://api.lantmateriet.se/distribution/geodatakatalog/sokning/v1/detaljplan/v2.Error details, 900902Missing CredentialsRequired OAuth credentials not provided. Make sure your API invocation call has a header: 'Authorization : Bearer ACCESS_TOKEN'
2022-02-12T09:00:13 INFO Fetching collections...
2022-02-12T09:00:13 CRITICAL Problem in fetching content for https://api.lantmateriet.se/distribution/geodatakatalog/sokning/v1/detaljplan/v2.Error details, 900902Missing CredentialsRequired OAuth credentials not provided. Make sure your API invocation call has a header: 'Authorization : Bearer ACCESS_TOKEN'
Add support for the plugin to be able to store operations information and errors into local log file and remote log files.
For logs submitted remotely, add option for user to be able to enable/disable the feature.
Use python logging package https://docs.python.org/3/library/logging.html and QgsMessageLog class to make sure the logs appear in QGIS log messages panel, after they have been logged.
For the moment, the GitHub page index is redirecting to 404.html
See the branch gh-pages
Add project skeleton for the initial plugin development, skeleton can be constructed from a template provided by plugin builder https://plugins.qgis.org/plugins/pluginbuilder3/.
Add a button to the Results tab which will add footprints for all items to QGIS. Ideally these would come as one layer with multiple features in that layer - but that may be a bit more tricky.
Setup infrastructure for testing in the plugin, the test suite should cover plugin functional features and include a mockup stac-api server for testing plugin functionality against stac-api spec.
The setup should enable running tests against QGIS LTR, QGIS nightly and QGIS 3.22 versions. See https://github.com/qgis/QGIS/tree/master/.docker/qgis_resources/test_runner for reference in running plugin test inside QGIS application.
Implement support for downloading STAC item assets via the download assets combo box that is available in the search result item widget.
Once the user have created a new connection we need to save the connection credential including the authentication configurations into the QGIS settings.
The plugin settings manager added in #13 is responsible for this operation.
Also we need to make sure the connection widget in the "Search" tab respond accordingly to the create, edit and delete actions on the connections.
The expected behavior is when a new connection is added, the connection should be active and selected from the connections list, when connection is deleted the last added connection should be active and selected.
Currently when adding a new connection, the connection is not persisted hence not shown in the connections list combo box.
Might be related to #16
screenshot
Investigate previous QGIS STAC plugin from Radiant Earth to see if code contains useful items for this project
if there are no settings.value("created_date") the plugin can't get Connection Setting
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.