Code Monkey home page Code Monkey logo

open-science-catalog-metadata's Introduction

open-science-catalog-metadata

About Open Science Catalog

The Open Science Data Catalog (https://opensciencedata.esa.int) is an ESA Open Science activity aiming to provide enhance the discoverability and use of the various scientific and value-added results (i.e. data, code, documentation) achieved in Earth System Science research activities funded by ESA Earth Observation. The Open Science Data Catalog provides open access for the scientific community to geoscience products (based on EO data from ESA and non-ESA missions and other geospatial information and models) across the whole spectrum of Earth Science domains. The Open Science Data Catalog adheres to FAIR principles and promotes reproducibility of scientific studies. The Open Science Data Catalog makes use of various Open-Source geospatial technologies such as pycsw, PySTAC, and OpenLayers and tries to contribute back to these projects in terms of software and standardisation.

Discover Open Science Data Catalog and access products and project's description at: Open Science Data Catalog.

Open Science Data Catalog documentation contains:

Functionalities

  • Provide contribution to the ESA EO Open Science framework
  • Catalogue of geoscience products, datasets and resources output by scientific research Projects funded by ESA EO
  • Discovery and access for geospatial products
  • Unified metadata across heterogeneous sources
  • Common dictionary
  • Discovery & Access to data + documentation (+ code)
  • Open to community curation & contribution
  • Synoptic view for EO gap analysis

This repository

This repository holds metadata all themes, variables, projects and products discoverable by Open Science Catalog Frontend. Each update to metadata is handled via Pull Request. This pull request allows for reviewers to see the changes to be applied in advance and to provide reviews as comments. If appropriate, the changes can be merged with the main branch of the repository. When a Pull Request is merged, the continuous integration pipeline is run, which triggers the building of a STAC catalog representing the Open Science Data Catalogue on the Static Catalog component.

The design document describes the architecture of Open Science Catalog and its design: https://docs.google.com/document/d/144d-ACMG_AzWtP6HohR1A7ltHGNY6NkhWmH9EyJb_vA/

To add a new product, the metadata corresponding to data set to be uploaded to the catalogue must be provided in a STAC .json file format.

open-science-catalog-metadata's People

Contributors

open-science-data avatar silvester-pari avatar constantinius avatar edobrowolska avatar kalxas avatar dedop-cotton avatar veronicagonzalezg avatar

Stargazers

Juanjo Bazán avatar Anca Anghelea avatar Pete Gadomski avatar

Watchers

Bernhard Mallinger avatar  avatar  avatar Fabrice Brito avatar Anca Anghelea avatar

open-science-catalog-metadata's Issues

SeasFire-SeasFireCube_v3

Datacube available in OSC [Developer version]
https://staging.opensciencedata.esa.int/products/seasfire-cube/collection

Dataset to be added to existing project:
https://github.com/EOEPCA/open-science-catalog-metadata/tree/3f96d77abe65d3afb477cf463e4402089956f4d5/projects/seasfire-earth-system-deep-learning-for-seasonal-fire-forecasting-in-europea

Link to datacube and code (python tutorial to visualize the cube) available in zenodo: https://zenodo.org/record/8055879
Received on: 23/08/2023
Project PI: [email protected]

Actions to take:

  • add variable to product

  • create product page on staging

  • import datacube to S3 bucket to enable discovery by user

  • convert the datacube .zarr to stac item using stactools-datacube package

  • ingest stac item to existing project

4DGreenland - Supraglacial Storage and Drainage - Lake Features Mapped by Sentinel-1

AlpGlacier-Snow Cover Product

Dataset to be added to existing project:
https://github.com/EOEPCA/open-science-catalog-metadata/blob/3f96d77abe65d3afb477cf463e4402089956f4d5/projects/alpglacier/collection.json

Snow Cover Product will be published in 2024
Received on: 22/08/2023

Actions to take:

  • send reminder to data owners at the beginning of the next year (January 2024) to get Snow Cover Product
  • get access to Product once publicly available
  • ingest the Product to existing project

Data providers: [email protected] ; [email protected]

4DAntarctica - experimental database

Project's github page: open-science-catalog-metadata/projects/4d-antarctica at main · EOEPCA/open-science-catalog-metadata (github.com)

Cryosat - antarctic sea - SIT products

Dataset description: "Sea ice thickness derived from CryoSat-2 estimates of radar freeboard combined with the Altimetric Snow Depth (ASD) snow solution. ASD utilises data from SARAL AltiKa and CryoSat-2 to estimate the depth of snow on sea ice floes."
Variables of interest are:
snow_depth = depth of snow on sea ice floes


These variables need to be added to OSC
freeboard_radar = radar freeboard as measured by CryoSat-2
freeboard_ice = the ice freeboard, estimated by accounting for the speed of propagation of the radar pulse through the snow layer
SIT = sea ice thickness
It should be clear on the catalogue page that this product is monthly, and gridded on the EASE2 12.5km equal-area grid

AlpGlacier-Lake Product

Dataset to be added to existing project:
https://github.com/EOEPCA/open-science-catalog-metadata/blob/3f96d77abe65d3afb477cf463e4402089956f4d5/projects/alpglacier/collection.json

Lake Product will be published in 2024
Received on: 22/08/2023

Actions to take:

  • send reminder to data owner at the beginning of the next year (January 2024) to get two Lake Product)
  • get access to Product once publicly available
  • ingest the Product to existing project

Data providers: [email protected] ; [email protected]

Polar+ Ice Shelf-Annual mass budget of Antarctic ice shelves from 1997 to 2021

Dataset to be added to existing project:
https://github.com/EOEPCA/open-science-catalog-metadata/tree/3f96d77abe65d3afb477cf463e4402089956f4d5/projects/polar-ice-shelf

Link to Product collection of items: https://zenodo.org/record/8052519

Received on 02/08/2023
Publication forseen in September 2023

Actions to take:

Contact point: [email protected]

Arctic Sea Surface Salinity L2 orbits and L3 maps (V.3.1) [Dataset]

Product exists on OSC: https://github.com/EOEPCA/open-science-catalog-metadata/blob/main/products/sea-surface-salinity-arctic-salinity/collection.json

Full dataset available here: https://digital.csic.es/handle/10261/219679?mode=simple

  • project description updated: #167 (comment)
  • product documentation to update: DOI: https://doi.org/10.13140/RG.2.2.12195.58401 and: https://doi.org/10.5194/essd-14-307-2022
  • download products and move to s3bucket to enable data exploration in native cloud environment
  • create STAC collection from the products
  • @constantinius when I go from the product in OSC and want to visualize the product in github from Suggest changes button I get "File not found error" while I can see the product collection.json file when accessing it direclty from Github.. can you check that?

Reference project id/path instead of project name in product info

Example: https://github.com/EOEPCA/open-science-catalog-metadata/blob/e924e19e29db000f74dc55dca2f25559c625358a/data/products/product-84.json#L13

Is actually a reference to: https://github.com/EOEPCA/open-science-catalog-metadata/blob/e924e19e29db000f74dc55dca2f25559c625358a/data/projects/project-55.json

This field should really reference the project by id/path, instead of providing a string, so it can be properly linked from the product page.

Current status:
image

Goal:
image

Cryosat+ Antarctic Ocean - CS+AO Experimental Dataset

Project description: http://cryosat.mssl.ucl.ac.uk/csao/about-right.html

Access to products: http://cryosat.mssl.ucl.ac.uk/csao/public-docs.html

Received on 30/03/2023

Contact person: [email protected]

Actions to take:

Project json
https://github.com/EOEPCA/open-science-catalog-metadata/blob/main/projects/cryosat-antarctic-ocean/collection.json

4D-ANTARCTICA- Ice velocities

Product available at: https://cryoportal.enveo.at/data_files/2021-ESACCI-L3C-AIS-IV-S1-1M_200m-fv1.1_20220420.zip

Antarctic Ice Sheet monthly ice velocity - 2021
Variable: ice_velocity
Release Date, Version | 2022-04-20, 1
Descritpion: This dataset contains monthly gridded ice velocity maps of the Antarctic Ice Sheet derived from Sentinel-1 data acquired between 2021-01-01 and 2021-12-31. It was generated by ENVEO, as part of the ESA Antarctic Ice Sheet Climate Change Initiative project (Antarctic_Ice_Sheet_cci). The surface velocity is derived by applying feature tracking techniques using Sentinel-1 synthetic aperture radar (SAR) data acquired in the Interferometric Wide (IW) swath mode. Ice velocity is provided at 200m grid spacing in Polar Stereographic projection (EPSG: 3031). The horizontal velocity components are provided in true meters per day, towards easting and northing direction of the grid. The vertical displacement is derived from a digital elevation model. Provided is a NetCDF file with the velocity components: vx, vy, vz, along with maps showing the magnitude of the horizontal components, the valid pixel count and uncertainty. The product combines all ice velocity maps, based on 6- and 12-day repeats, acquired within a single month in a monthly averaged product.
Temporal coverage: 2021-01-01 00:00:00.000000 to 2021-12-31 00:00:00.000000
Extent: POLYGON ((-180 -60, 180 -60, 180 -90, -180 -90, -180 -60))

4DAntarctica-Supraglacial lakes and channels in West Antarctica and Antarctic Peninsula during January 2017 (v.1.1)

Product already exists in OSC: https://github.com/EOEPCA/open-science-catalog-metadata-staging/blob/291acc52b0863fd29f28b727e02965f5d55fd6a1/products/supraglacial-lakes-west-4dantarctica/collection.json

Access data: https://zenodo.org/record/5642755#.ZDa5z3bP1aR

  • Only need WAIS_Max_Extent.zip (data then available as shapefile, or geojson, or kmz)
  • create stac item
  • place to s3 bucket to make it discoverable

ALBATROSS datasets

project
https://github.com/EOEPCA/open-science-catalog-metadata/tree/3f96d77abe65d3afb477cf463e4402089956f4d5/projects/albatross-altimetry-for-bathymetry-and-tideretrievals-for-the-southern-ocean-sea-ice-and-ice-shelves

eo4society
https://eo4society.esa.int/projects/albatross/

products list from the website
https://albatross.noveltis.fr/products_/

e-mail to the team
[email protected]

List of products published on the project's website:

  • Tidal elevation along CryoSat-2 track during the year 2019

  • Tidal elevation and current along trajectories of drifter from the NOAA Global Drifter Program

  • Tidal elevation at the Brunt Halley and Filchner Ronne (R145 and E5) GPS stations

  • Tidal elevation at tide gauges in the Kerguelen Islands (8 files at different position and time)

  • Tidal amplitude and phase for K1, K2, M2, N2, O1, P1, Q1, S2 tidal constituents for 54 tide gauges from the Antarctic Tide Gauge Database

  • contact team to get access to products

Cryosat+ Antarctic Ocean - Sea Ice Product (no SIT): 2010-2013

Products available at: http://cryosat.mssl.ucl.ac.uk/csao/fb_sla_cs2_sam_2010_NOSIT.nc
http://cryosat.mssl.ucl.ac.uk/csao/fb_sla_cs2_sam_2011_NOSIT.nc
http://cryosat.mssl.ucl.ac.uk/csao/fb_sla_cs2_sam_2012_NOSIT.nc
http://cryosat.mssl.ucl.ac.uk/csao/fb_sla_cs2_sam_2013_NOSIT.nc

4 product could be added in one page in stac catalog since they reflect the same variable and the same extent

  • create product page
  • add products to s3 bucket
  • create stac catalog (4 items)

AlpGlacier-Slope instabilities

Dataset to be added to existing project:
https://github.com/EOEPCA/open-science-catalog-metadata/blob/3f96d77abe65d3afb477cf463e4402089956f4d5/projects/alpglacier/collection.json

Slope instabilities will be delivered by 30/09/2023
Received on: 22/08/2023

Actions to take:

  • send reminder to data owner on 30/09/2023 to get the Product (Slope)
  • get access to Product once publicly available
  • ingest the Product to existing project

Data providers: [email protected] ; [email protected]

Cryosat+ Antarctic Ocean - Sea Ice Product: 2013-2020

SOFresh - Final SSS Product

Project description:
https://eo4society.esa.int/projects/so-fresh/

Link to product provided:
https://drive.google.com/file/d/1SbLKx6pSj6XaiO0nBxhijnsWnDir0jXW/view?usp=sharing
https://drive.google.com/file/d/1v8kYIIrZFPBP8GBidKUkSCtN5kY19LCg/view?usp=sharing
https://drive.google.com/file/d/1xWU7CtuvIqJBTkKl263rpXZ9IuYADRW3/view?usp=sharing
https://drive.google.com/file/d/106CKHJ8qjwSa3EsqAkkcpV009X13eTbT/view?usp=sharing
https://drive.google.com/file/d/1UuyFxwIXiI8rIlJrU9PPEoe7RaBEwyyd/view?usp=sharing
https://drive.google.com/file/d/1EBCYn6FmozbGzXpjqS86P2i3Gy_2fU4g/view?usp=sharing
https://drive.google.com/file/d/1YdUTR4U2Waou72KrysUDbg3YLBqQibVs/view?usp=sharing
https://drive.google.com/file/d/1TtihRRAVxmFZ9jdVTDVR7Hgz_X9BFpy2/view?usp=sharing
https://drive.google.com/file/d/1FvUAZTGVRLbeN1xms1Eo_SRda6oEBAQi/view?usp=sharing
https://drive.google.com/file/d/1BviiUWmI5j3toXZXDF9FPkTa0yG-jPl9/view?usp=sharing
https://drive.google.com/file/d/1rQAISIYkN9B9RDxCz5pIFqkRyG8uqMlX/view?usp=sharing
https://drive.google.com/file/d/1iFG1fi-rhkt71mzVewnPfA5vp4Wo5q90/view?usp=sharing

Received on 02/08/2023
Association of DOI and Publications still pending. Minor changes to dataset are expected in September 2023

Contact person: [email protected] Project PI: [email protected]
Actions to take:

4DGreenland - experimental dataset

Project's github page: https://github.com/EOEPCA/open-science-catalog-metadata/tree/3f96d77abe65d3afb477cf463e4402089956f4d5/projects/4d-greenland

Experimental dataset collection repository: https://data.dtu.dk/collections/ESA_POLAR_4D_Greenland_Experimental_Dataset_Collection/6080925/1

Import 5 missing products to OSC:

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.