Comments (7)
tomkralidis
commented on July 28, 2024
@alexandreleroux questions/comments:
- what's the value proposition around moving to YAML?
- we can certainly do more with YAML as a config format (nesting, better includes, cardinality, etc.)
- should we support both, or just a hard move to YAML (with a migration script for MCF to YAML)? If we want to support both, then then YAML possibilities are bound to the MCF (i.e. the ini file) limitations
- here's a sample YAML representation of https://github.com/geopython/pygeometa/blob/master/sample.mcf
- metadata
identifier: 3f342f64-9348-11df-ba6a-0014c2c00eab
language: en
language_alternate: fr
charset: utf8
parentidentifier: someparentid
hierarchylevel: dataset
datestamp: 2014-11-11
dataseturi: http://some/minted/uri
- spatial
datatype: vector
geomtype: point
crs: 4326
bbox: -141,42,-52,84
- identification
language: eng; CAN
charset: utf8
title_en: title in English
title_fr: title in French
abstract_en: abstract in English
abstract_fr: abstract in French
keywords_en: kw1 in English,kw2 in English,kw3 in English
keywords_fr: kw1 in French,kw2 in French,kw3 in French
keywords_wmo: FOO,BAR
keywords_type: theme
keywords_gc_cst_en: kw1,kw2
keywords_gc_cst_fr: kw1,kw2
topiccategory: climatologyMeteorologyAtmosphere
publication_date: 2000-09-01T00:00:00Z
fees: None
accessconstraints: otherRestrictions
rights_en: Copyright (c) 2010 Her Majesty the Queen in Right of Canada
rights_fr: Copyright (c) 2010 Her Majesty the Queen in Right of Canada
url: http://geogratis.ca/geogratis/en/product/search.do?id=08DB5E85-7405-FE3A-2860-CC3663245625
temporal_begin: 1950-07-31
temporal_end: now
status: onGoing
maintenancefrequency: continual
- contact:main
organization: Environment Canada
url: http://www.ec.gc.ca/
individualname: Tom Kralidis
positionname: Senior Systems Scientist
phone: +01-123-456-7890
fax: +01-123-456-7890
address: 4905 Dufferin Street
city: Toronto
administrativearea: Ontario
postalcode: M3H 5T4
country: Canada
email: [email protected]
hoursofservice: 0700h - 1500h EST
contactinstructions: email
- contact:distribution
#ref: contact:main
organization: Environment Canada
url: http://www.ec.gc.ca/
individualname: Tom Kralidis
positionname: Senior Systems Scientist
phone: +01-123-456-7890
fax: +01-123-456-7890
address: 4905 Dufferin Street
city: Toronto
administrativearea: Ontario
postalcode: M3H 5T4
country: Canada
email: [email protected]
hoursofservice: 0700h - 1500h EST
contactinstructions: email
- distribution:waf
url: http://dd.meteo.gc.ca
type: WWW:LINK
name: my waf
description_en: description in English
description_fr: description in French
function: download
- distribution:wms
url: http://dd.meteo.gc.ca
type: OGC:WMS
name_en: roads
name_fr: routes
description_en: description in English
description_fr: description in French
function: downloadfrom pygeometa.
alexandreleroux
commented on July 28, 2024
My non-expert understanding:
- yaml's indentation makes it much easier to read than the mcf / ini format
- this is significant given that's what data stewards in an organisation leveraging pygeometa will use to view and edit metadata records
- like you pointed out already, yaml and its libraries support nesting / includes / cardinality
- I'm fine with supporting only yaml in the future. A tool to migrate from mcf to yaml would indeed be nice for existing pygeometa users, but not necessary?
Have I forgot or am I wrong on anything? Thx
from pygeometa.
alexandreleroux
commented on July 28, 2024
@tomkralidis in your YAML-MCF example, is there a reason both Contact and Distribution are split into distinct yaml elements? Example, why not, for distribution:
- distribution:
- waf
url: http://dd.meteo.gc.ca
type: WWW:LINK
name: my waf
description_en: description in English
description_fr: description in French
function: download
-wms
url: http://dd.meteo.gc.ca
type: OGC:WMS
name_en: roads
name_fr: routes
description_en: description in English
description_fr: description in French
function: downloadSimilar comment for keywords under identification, could they be grouped together?
- identification
language: eng; CAN
charset: utf8
title_en: title in English
title_fr: title in French
abstract_en: abstract in English
abstract_fr: abstract in French
keywords:
keywords_en: kw1 in English,kw2 in English,kw3 in English
keywords_fr: kw1 in French,kw2 in French,kw3 in French
keywords_wmo: FOO,BAR
keywords_type: theme
keywords_gc_cst_en: kw1,kw2
keywords_gc_cst_fr: kw1,kw2Does such groupings make sense? Thx! -- Alex
from pygeometa.
tomkralidis
commented on July 28, 2024
Makes sense, and we can apply it to different sections as well. This will be nice as it will allow clean grouping that wasn't possible before given the ini format.
Updated example:
metadata:
identifier: 3f342f64-9348-11df-ba6a-0014c2c00eab
language: en
language_alternate: fr
charset: utf8
parentidentifier: someparentid
hierarchylevel: dataset
datestamp: 2014-11-11
dataseturi: http://some/minted/uri
spatial:
datatype: vector
geomtype: point
crs: 4326
bbox: -141,42,-52,84
identification:
language: eng; CAN
charset: utf8
title_en: title in English
title_fr: title in French
abstract_en: abstract in English
abstract_fr: abstract in French
keywords:
default:
keywords_en: [kw1 in English,kw2 in English,kw3 in English]
keywords_fr: [kw1 in French,kw2 in French,kw3 in French]
wmo:
keywords_en: [FOO,BAR]
keywords_type: theme
gc_cst:
keywords_en: [kw1,kw2]
keywords_fr: [kw1,kw2]
topiccategory: climatologyMeteorologyAtmosphere
publication_date: 2000-09-01T00:00:00Z
fees: None
accessconstraints: otherRestrictions
rights_en: Copyright (c) 2010 Her Majesty the Queen in Right of Canada
rights_fr: Copyright (c) 2010 Her Majesty the Queen in Right of Canada
url: http://geogratis.ca/geogratis/en/product/search.do?id=08DB5E85-7405-FE3A-2860-CC3663245625
temporal_begin: 1950-07-31
temporal_end: now
status: onGoing
maintenancefrequency: continual
contact:
main: &contact_main
organization: Environment Canada
url: http://www.ec.gc.ca/
individualname: Tom Kralidis
positionname: Senior Systems Scientist
phone: +01-123-456-7890
fax: +01-123-456-7890
address: 4905 Dufferin Street
city: Toronto
administrativearea: Ontario
postalcode: M3H 5T4
country: Canada
email: [email protected]
hoursofservice: 0700h - 1500h EST
contactinstructions: email
distribution: *contact_main
distribution:
waf:
url: http://dd.meteo.gc.ca
type: WWW:LINK
name: my waf
description_en: description in English
description_fr: description in French
function: download
wms:
url: http://dd.meteo.gc.ca
type: OGC:WMS
name_en: roads
name_fr: routes
description_en: description in English
description_fr: description in French
function: downloadfrom pygeometa.
tomkralidis
commented on July 28, 2024
@alexandreleroux for review and comment: Initial changes are in https://github.com/geopython/pygeometa/tree/issue-54. Notes:
- see the updated
sample.mcffor updates to the config representation now in YAML- contacts are now dicts
- distributions are now dicts
- we can now use YAML's native reference/dereferencing features (
&ref,*ref) instead of arefoption to reuse other parts of the YAML config
- HNAP based keywords are now just sections of the main
keywordsgroup (keys calledhnap_category_information,hnap_category_geography,hnap_category_content, etc.)
from pygeometa.
tomkralidis
commented on July 28, 2024
@alexandreleroux more updates in branch issue-54:
- MCF now has a section called
mcf, which initially has aversionfield to denote the version of the MCF - the cli has been refactored to support a single
pygeometascript with subcommands. Examples:
$ pygeometa
Usage: pygeometa [OPTIONS] COMMAND [ARGS]...
Options:
--help Show this message and exit.
Commands:
generate_metadata
migrate
$ pygeometa generate_metadata --help
Usage: pygeometa generate_metadata [OPTIONS]
Options:
--mcf PATH Path to metadata control file (.yml)
--output FILENAME Name of output file
--schema [wmo-cmp|iso19139|iso19139-hnap]
Metadata schema
--schema_local DIRECTORY Locally defined metadata schema
--help Show this message and exit.
$ pygeometa generate_metadata --mcf=path/to/file.yml --schema=iso19139 # to stdout
# migrate an old MCF file to YAML
$ pygeometa migrate --mcf=path/to/file.mcf- added migration script to convert old MCF files to YAML
from pygeometa.
tomkralidis
commented on July 28, 2024
Implemented in master.
from pygeometa.
Related Issues (20)
- roundtrip iso-mcf-iso lacks crs
- Add license restriction to iso19139 HOT 3
- Error in using Transform function to convert iso19139 to wmo-wcmp2 HOT 4
- owslib importer for ISO is missing the capture of some attributes during parsing HOT 3
- "AttributeError: 'dict' object has no attribute 'type'" when running pygeometa metadata import for iso19139 schema HOT 5
- Pygeometa Does Not Automatically Populate Extent Metadata for WCMP2 Record from MCF HOT 2
- contacts and distributions as array? HOT 1
- conversion of sample yaml to stac throws error HOT 1
- import metadata not activated on demo.pygeoapi.io
- Add support for importing QGIS qmd files HOT 2
- get schemas method on metadata processing plugin pygeoapi HOT 1
- check thesaurus import from iso19139 HOT 2
- alternate relations then parentidentifier
- add schema.org schema
- attributes and dimensions alignment vs featuretypes
- failing test test metadata import HOT 1
- stac generation fails if mcf has no 'pointofcontact'
- crs has no prefix?
- Support FGDC XML Metadata Format?
- iso19139 export throws error if extent is None HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pygeometa.