zdefne-usgs / ocean-iso-metadata Goto Github PK

I think we have already captured this somewhere in this repository but cannot seem to find it now:

I believe my plan was to have one master Udunits file: udunits2_combined_whmsc.xml

and a trail of versioned Udunit files (e.g. udunits2_combined_whmsc_v3.xml)

The meaning of _v2, _v3, etc:
The higher version number means most recent. And the master file should be the same as the highest version. Master file is the one that should be copied to the same location with a user's NCML file. This is the default location in used coded in our custom XSL file that is used by ncISO.

This issue should stay open once I verify that we have all the versions of Udunits file in the Units folder and the master file is same with the highest version.

Horizontal (geospatial_bounds_crs)

The coordinate reference system (CRS) of the point coordinates in the geospatial_bounds attribute. This CRS may be 2-dimensional or 3-dimensional, but together with geospatial_bounds_vertical_crs, if that attribute is supplied, must match the dimensionality, order, and meaning of point coordinate values in the geospatial_bounds attribute. If geospatial_bounds_vertical_crs is also present then this attribute must only specify a 2D CRS. EPSG CRSs are strongly recommended. If this attribute is not specified, the CRS is assumed to be EPSG:4326. Examples: 'EPSG:4979' (the 3D WGS84 CRS), 'EPSG:4047'.

In ISO 19115-2: The MD_CRS from ISO 19111 is not included in ISO 19115

Vertical (geospatial_bounds_vertical_crs)

The vertical coordinate reference system (CRS) for the Z axis of the point coordinates in the geospatial_bounds attribute. This attribute cannot be used if the CRS in geospatial_bounds_crs is 3-dimensional; to use this attribute, geospatial_bounds_crs must exist and specify a 2D CRS. EPSG CRSs are strongly recommended. There is no default for this attribute when not specified. Examples: 'EPSG:5829' (instantaneous height above sea level), "EPSG:5831" (instantaneous depth below sea level), or 'EPSG:5703' (NAVD88 height).

In ISO 19115-2: /gmi:MI_Metadata/gmd:identificationInfo/gmd:MD_DataIdentification/gmd:extent/gmd:EX_Extent/gmd:verticalElement/gmd:EX_VerticalExtent/gmd:verticalCRS

These need to be included in the final checks to make sure the USGS data releases are complete:

1. The reviewers should catch most things, but these are some of the things that I am spot checking:
2. The metadata date in gmd:dateStamp. This should be pretty recent to when the data release is checked, not 2 years ago.
3. An FSP distribution liability statement in gmd:MD_LegalConstraints.
4. Present and valid USGS Thesaurus terms in gmd:descriptiveKeywords.
5. If the data are released by ScienceBase, then the distributor should be ScienceBase in gmd:distributionInfo
6. I'm also checking to see if the element gmd:metadataMaintenance is present. Folks usually put in there what was used to create the metadata. And then we can further use that element when edits are made to the metadata to act as sort of a revision history.

Should reflect the info for the staff pages, and the distributor info page.

<gmd:protocol>
     <gco:CharacterString>WWW:LINK</gco:CharacterString>
</gmd:protocol>
<gmd:applicationProfile>
    <gco:CharacterString>web browser</gco:CharacterString>
</gmd:applicationProfile>
<gmd:name>
    <gco:CharacterString>
        <xsl:value-of select="$urlName"/>
    </gco:CharacterString>
</gmd:name>
<gmd:description>
    <gco:CharacterString>
        <xsl:value-of select="$urlDescription"/>
    </gco:CharacterString>
</gmd:description>

To handle some issues Zafer brought up with the ncISO utility ; the latest version is 2.3.9

Thanks for letting me know. All of the XSL and the Java code originates from work done by @dneufeldcu and colleagues. I am just the caretaker. I made 2.3.9 official.

NOAA-PMEL/uafnciso#21

Because I am installing a new OS, may be in instructions, we should have way to install python and dependencies.

This is one way to install python dependency:

https://www.digitalocean.com/community/tutorials/how-to-install-anaconda-on-ubuntu-18-04-quickstart

How and where to define "Georeferencable" in the ISO metadata?

For example, http://cfconventions.org/Data/cf-conventions/cf-conventions-1.7/cf-conventions.html
or CF github page
https://github.com/cf-convention/cf-conventions/releases
has the information we need.

Ideally CF conventions should have a DOI number that we can cite in the metadata. Where do we insert that information in the ISO metadata?

yaml (add a comment for include and exclude)

• license:
  to be used for "Please cite this as..."

                    <xsl:if test="count($license)">
                        <gmd:resourceConstraints>
                            <gmd:MD_LegalConstraints>
                                <gmd:useLimitation>
                                    <gco:CharacterString>
                                        <xsl:value-of select="$license[1]"/>
                                    </gco:CharacterString>
                                </gmd:useLimitation>
                            </gmd:MD_LegalConstraints>
                        </gmd:resourceConstraints>
                    </xsl:if>

• CMG_Portal additional info (link, description, protocol etc..)
• crosReference a sciencePaper
• verticalExtent
• institution to handle organisationName

	<gmd:organisationName>
		<gco:CharacterString>U.S. Geological Survey, Woods Hole Coastal and Marine Science Center</gco:CharacterString>
	</gmd:organisationName>

• add fields for principalInvestigator?

Based on the comparison with the xml generated with ufncISO:
java -Xms1024m -Xmx1024m -jar ncISO-2.3.5.jar -ts http://geoport-dev.whoi.edu/thredds/sand/usgs/Projects/BBLEH/run071tRX/catalog.xml -num 1 -depth 20 -iso true

1
There are a couple of major differences between the ISO metadata generated with the Thredds ISO service and the ISO metadata generated with the ncISO, such as the fileIdentifier and the dimensions.

As reading the recommendations looks like one should combine the naming authority and the id to put back into the id field. The combination of the "naming authority" and the "id" should be a globally unique identifier for the dataset. It is recommended that the combination should be unique, but they are separate fields. ncISO does not do it either (or is it because Thredds server already automatically combine the two to get the unique identifier?).

2
Treatment of dimensions has changed in ncISO. See Dimension_Information. In addition to the temporal, vertical and topoloigcal dimensions there are now dimensions for column, row, tracers, boundary, xi-, eta- (rho, psi) and s-(w, rho).

3
Thredds netcdf http service is missing in ncISO and the netcdf subset service definition is simplified (has less fields). The solution may be using ncISO, but need find out why http service is not described.

4
However, in neither of them MD_Georeferenceable spatial representation is not automatically generated.

the xml was coding as a "revision" not "publication". Is this hard coded? Is there something I could have done besides changing it by hand in the xml?

not content_coverage_type

yaml2ncml need to be updated to change the wrong entries:

    <ncml:attribute name="content_coverage_type" value="modelResult" />

Is the .xsl stylesheet the most updated version? I got another copy from John Warner today and wanted to make sure they were the same on this page. Just wanted to help update this utility.

I find that the axisDimensionProperties always misses the vertical resolution for my dataset.

     <gmd:axisDimensionProperties>
        <gmd:MD_Dimension>
           <gmd:dimensionName>
              <gmd:MD_DimensionNameTypeCode codeList="http://www.ngdc.noaa.gov/metadata/published/xsd/schema/resources/Codelist/gmxCodelists.xml#MD_DimensionNameTypeCode"codeListValue="vertical">vertical</gmd:MD_DimensionNameTypeCode>
           </gmd:dimensionName>
           <gmd:dimensionSize>
              <gco:Integer>16</gco:Integer>
           </gmd:dimensionSize>
           <gmd:resolution gco:nilReason="missing"/>.   <------MISSING
        </gmd:MD_Dimension>
     </gmd:axisDimensionProperties>

so I hardcoded it into my version of the xsl stylesheet (around line 300)

<xsl:if test="count($verticalVariableName)">
<xsl:call-template name="writeDimension">
<xsl:with-param name="dimensionType" select="'vertical'"/>
<xsl:with-param name="dimensionUnits" select="'meter'"/> <---HERE



<xsl:with-param name="dimensionResolution" select=" '0.01' "/>. <--- HERE
<xsl:with-param name="dimensionSize" select="/nc:netcdf/nc:dimension[contains(@name,$verticalVariableName)]/@Length"/>
</xsl:call-template>
</xsl:if>

I did the same thing for the temporal dimension

                        <xsl:if test="count($timeVariableName)">
                            <xsl:call-template name="writeDimension">
                                <xsl:with-param name="dimensionType" select="'time'"/>
                                <!--<xsl:with-param name="dimensionUnits" select="$temporalUnits[1]"/>-->
                                <xsl:with-param name="dimensionUnits" select="'seconds'"/>
                                <xsl:with-param name="dimensionResolution" select=" '3600' "/>
                                <!--<xsl:with-param name="dimensionResolution" select="$timeResolution[1]"/-->>
                                <xsl:with-param name="dimensionSize" select="/nc:netcdf/nc:dimension[contains(@name,$timeVariableName)]/@length"/>
                            </xsl:call-template>
                        </xsl:if>

I imagine this could be hard coded into the yaml file, however I am finding it hard to figure out the workflow for the spatial representation info.

I think it would be good to include within the /project_name/ folder where the .ncml and .nc files should be so that the command line for ncml -> iso works as intended. This should also include where the units stylesheet should go.

Links for unit descriptions do not work properly. Used a combination of following links t o fix.
8 occurrences of:
https://www.unidata.ucar.edu/software/udunits/udunits-2.2.25/udunits/udunits2-base.xml
16 occurrences of:
https://www.unidata.ucar.edu/software/udunits/udunits-current/udunits/udunits2-common.xml
3 occurrences of:
https://www.unidata.ucar.edu/software/udunits/udunits-2.2.25/udunits/udunits2-derived.xml
1 occurrence of:
https://www.unidata.ucar.edu/software/udunits/udunits-2.2.25/udunits/udunits2-accepted.xml
11 occurrences of:
http://www.xml-cml.org/unit/si/
15 occurrences of:
http://www.xml-cml.org/unit/nonSi/

However, rubric does not like the links that do not refer to an xml file.

Maybe I was doing something wrong, but I had to add all of the input files files to the xml by hand. If it is, in fact, that I was doing something wrong, perhaps the wiki (https://github.com/zdefne-usgs/ocean-iso-metadata/wiki/1.-Publishing-hydrodynamic-model-output) could be updated to show how input files should be included?

Where to keep the most recent version of the udunits.xml file? It is probably the best to have it versioned _v1, _v2, etc and added to the Units folder in this repo? I thought I'd already decided on this. Maybe I did and decided that was not the best option, need to think about it again. Right now each data release has a copy of the latest version in the thredds folder for that specific run

udunits2_combined_CMHRP.xml

Maybe include place keywords from USGS Thesaurus in addition to ISO keywords?

• Can we please update to latest yaml file for example ?
• Replace staff link url