tdwg / bdq Goto Github PK
View Code? Open in Web Editor NEWBiodiversity Data Quality (BDQ) Interest Group
Home Page: https://github.com/tdwg/bdq
Biodiversity Data Quality (BDQ) Interest Group
Home Page: https://github.com/tdwg/bdq
My last name is incorrect in the wiki home page.
The correct is Veiga instead Viega.
TestField | Value |
---|---|
GUID | 431467d6-9b4b-48fa-a197-cd5379f5e889 |
Label | AMENDMENT_SCIENTIFICNAMEID_FROM_TAXON |
Description | Propose amendment to the value of dwc:scientificNameID if it can be unambiguously resolved from bdq:sourceAuthority using the available taxon terms. |
TestType | Amendment |
Darwin Core Class | Taxon |
Information Elements ActedUpon | dwc:scientificNameID |
Information Elements Consulted | dwc:taxonID |
dwc:acceptedNameUsageID | |
dwc:originalNameUsageID | |
dwc:taxonConceptID | |
dwc:scientificName | |
dwc:higherClassification | |
dwc:kingdom | |
dwc:phylum | |
dwc:class | |
dwc:order | |
dwc:superfamily | |
dwc:family | |
dwc:subfamily | |
dwc:tribe | |
dwc:subtribe | |
dwc:genus | |
dwc:genericName | |
dwc:subgenus | |
dwc:specificEpithet | |
dwc:infraspecificEpithet | |
dwc:cultivarEpithet | |
dwc:vernacularName | |
dwc:scientificNameAuthorship | |
dwc:taxonRank | |
Expected Response | EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if dwc:scientificNameID is not EMPTY or if all of dwc:scientificName, dwc:genericName, dwc:specificEpithet, dwc:infraspecificEpithet, dwc:scientificNameAuthorship, and dwc:cultivarEpithet are EMPTY, FILLED_IN the value of dwc:scientificNameID for an unambiguously resolved single taxon record in the bdq:sourceAuthority through (1) the value of dwc:scientificName or (2) if dwc:scientificName is EMPTY through values of the terms dwc:genericName, dwc:specificEpithet, dwc:infraspecificEpithet, dwc:scientificNameAuthorship and dwc:cultivarEpithet, or (3) if ambiguity produced by multiple matches in (1) or (2) can be disambiguated to a single Taxon using the values of dwc:subtribe, dwc:tribe, dwc:subgenus, dwc:genus, dwc:subfamily, dwc:family, dwc:superfamily, dwc:order, dwc:class, dwc:phylum, dwc:kingdom, dwc:higherClassification, dwc:taxonID, dwc:acceptedNameUsageID, dwc:originalNameUsageID, dwc:taxonConceptID, dwc:taxonomicRank, and dwc:vernacularName; otherwise NOT_AMENDED |
Data Quality Dimension | Conformance |
Term-Actions | TAXONID_FROM_TAXON |
Parameter(s) | bdq:sourceAuthority |
Source Authority | bdq:sourceAuthority default = "GBIF Backbone Taxonomy" {[https://doi.org/10.15468/39omei]} {API endpoint [https://api.gbif.org/v1/species?datasetKey=d7dddbf4-2cf0-4f39-9b2a-bb099caae36c&name=]} |
Specification Last Updated | 2023-09-17 |
Examples | [dwc:taxonID="", dwc:scientificNameID="", dwc:acceptedNameUsageID="", dwc:originalNameUsageID="", dwc:taxonConceptID="", dwc:scientificName="Chicoreus palmarosae (Lamarck, 1822)", dwc:higherClassification="", dwc:kingdom="Animalia", dwc:phylum="Mollusca", dwc:class="Gastropoda", dwc:order="", dwc:family="Muricidae", dwc:subfamily="", dwc:genus="Chicoreus", dwc:genericName="Chicoreus", dwc:subgenus="", dwc:infragenericEpithet="", dwc:specificEpithet="palmarosae", dwc:infraspecificEpithet="", dwc:cultivarEpithet="", dwc:vernacularName="", dwc:scientificNameAuthorship="(Lamarck, 1822)", dwc:taxonRank="", bdq:sourceAuthority=”marinespecies.org”: Response.status=FILLED_IN, Response.result=dwc:scientificNameID="urn:lsid:marinespecies.org:taxname:208134", Response.comment="dwc:scientificName matched to unique taxon record in WoRMS, exact match on name and authorship. Resolvable at https://marinespecies.org/aphia.php?p=taxdetails&id=208134"] |
[dwc:taxonID="", dwc:scientificNameID="", dwc:acceptedNameUsageID="", dwc:originalNameUsageID="", dwc:taxonConceptID="", dwc:scientificName="Graphis", dwc:higherClassification="", dwc:kingdom="", dwc:phylum="", dwc:class="", dwc:order="", dwc:family="", dwc:subfamily="", dwc:genus="", dwc:genericName="", dwc:subgenus="", dwc:infragenericEpithet="", dwc:specificEpithet="", dwc:infraspecificEpithet="", dwc:cultivarEpithet="", dwc:vernacularName="", dwc:scientificNameAuthorship="", dwc:taxonRank="": Response.status=NOT_AMENDED, Response.result="", Response.comment="dwc:scientificName="Graphis" is ambiguous as could be either a lichen or a gastropod."] | |
Source | FP-Akka |
References |
|
Example Implementations (Mechanisms) | FP-KurationServices, Arctos, MCZbase, Symbiota |
Link to Specification Source Code | |
Notes | Return a result with no value and a Result.status of NOT_AMENDED with a Response.comment of ambiguous if the information provided does not resolve to a unique result (e.g. if homonyms exist and there is insufficient information in the provided data, for example using the lowest ranking taxa in conjunction with dwc:dwc:scientificNameAuthorship, to resolve them). When referencing a GBIF taxon by GBIF's identifier for that taxon, use the the pseudo-namespace "gbif:" and the form "gbif:{integer}" as the value for dwc:scientificNameID. |
TestField | Value |
---|---|
GUID | 3c2590c7-af8a-4eb4-af57-5f73ba9d1f8e |
Label | AMENDMENT_COORDINATES_FROM_VERBATIM |
Description | Propose amendment to the values of dwc:decimalLatitude and dwc:decimalLongitude from information in the verbatim coordinates terms. |
TestType | Amendment |
Darwin Core Class | Location |
Information Elements ActedUpon | dwc:decimalLatitude |
dwc:decimalLongitude | |
Information Elements Consulted | dwc:verbatimCoordinates |
dwc:verbatimLatitude | |
dwc:verbatimLongitude | |
dwc:verbatimCoordinateSystem | |
dwc:verbatimSRS | |
Expected Response | INTERNAL_PREREQUISITES_NOT_MET if either dwc:decimalLatitude or dwc:decimalLongitude were not EMPTY, or either dwc:verbatimLatitude and dwc:verbatimLongitude, or dwc:verbatimCoordinates were not unambiguously interpretable into valid coordinates; FILLED_IN the values of dwc:decimalLatitude and dwc:decimalLongitude if unambiguous values can be interpreted from dwc:verbatimCoordinates or dwc:verbatimLatitude and dwc:verbatimLongitude, plus dwc:verbatimCoordinateSystem and dwc:verbatimSRS; otherwise NOT_AMENDED |
Data Quality Dimension | Completeness |
Term-Actions | COORDINATES_FROM_VERBATIM |
Parameter(s) | |
Source Authority | |
Specification Last Updated | 2023-09-17 |
Examples | [dwc:verbatimLatitude="-23.712", dwc:verbatimLongitude="", dwc:verbatimCoordinates="139.92", dwc:verbatimSRS="EPSG:4326", dwc:verbatimCoordinateSystem="decimal degrees": Response.status=FILLED_IN, Response.result=dwc:decimalLatitude="-23.712", dwc:decimalLongitude="139.923", dwc:geodeticDatum="EPSG:4326", Response.comment="Input fields contain interpretable values"] |
[dwc:verbatimLatitude="-23.712", dwc:verbatimLongitude="", dwc:verbatimCoordinates="", dwc:verbatimSRS="EPSG:4326", dwc:verbatimCoordinateSystem="decimal degrees", dwc:decimalLatitude="", dwc:decimalLongitude="": Response.status=NOT_AMENDED, Response.result="", Response.comment="dwc:verbatimLongitude is EMPTY"] | |
[dwc:verbatimLatitude="", dwc:verbatimLongitude="", dwc:verbatimCoordinates="54K 0390210 7377243", dwc:verbatimSRS="EPSG:32754", dwc:verbatimCoordinateSystem="decimal degrees", dwc:decimalLatitude="", dwc:decimalLongitude="": Response.status=AMENDED, Response.result={dwc:decimalLatitude="-23.712", dwc:decimalLongitude="139.923", dwc:geodeticDatum="EPSG:4326"}, Response.comment="dwc:verbatimLongitude is EMPTY"] | |
Source | ALA |
References |
|
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes |
TestField | Value |
---|---|
GUID | 39bb2280-1215-447b-9221-fd13bc990641 |
Label | AMENDMENT_DATEIDENTIFIED_STANDARDIZED |
Description | Propose amendment to the value of dwc:dateIdentified to a valid ISO date. |
TestType | Amendment |
Darwin Core Class | Identification |
Information Elements ActedUpon | dwc:dateIdentified |
Information Elements Consulted | |
Expected Response | INTERNAL_PREREQUISITES_NOT_MET if dwc:dateIdentified is EMPTY; AMENDED if the value of dwc:dateIdentified was not a properly formatted ISO 8601-1 date but was unambiguous and was altered to be a valid ISO 8601-1 date; otherwise NOT_AMENDED. |
Data Quality Dimension | Conformance |
Term-Actions | DATEIDENTIFIED_STANDARDIZED |
Parameter(s) | |
Source Authority | bdq:sourceAuthority = "ISO 8601-1:2019" {[https://www.iso.org/iso-8601-date-and-time-format.html]} |
Specification Last Updated | 2023-09-17 |
Examples | [dwc:dateIdentified="2021-28-10": Response.status=AMENDED, Response.result=dwc:dateIdentified="2021-10-28", Response.comment="dwc:dateIdentified assuming dwc:year, dwc:day and dwc:month"] |
[dwc:dateIdentified="21-10-28": Response.status=NOT_AMENDED, Response.result="", Response.comment="dwc:dateIdentified contains ambiguous values. It could be dd-mm-yy or yy-mm-dd"] | |
Source | Kurator |
References |
|
Example Implementations (Mechanisms) | Kurator:event_date_qc |
Link to Specification Source Code | https://github.com/FilteredPush/event_date_qc/blob/1abbd3f02eb6c28129764defab78f72156972864/src/main/java/org/filteredpush/qc/date/DwCEventDQ.java#L489 |
Notes | We reference Wikipedia for the ISO standard because the standard documents are not free. |
In a number of places - e.g. the Framework and in the Tests and Assertions, we have used "Duplicate" to indicate records that are equivalent when considering a subset of fields. This is used a lot is Species Distribution Modelling (SDM) where one record for a taxon in a grid square (i.e. only using taxon, latitude, longitude is regarded as a duplicate). In botany, a duplicate is where parts of a single collection from one plant is sent to a number of different institutions. Some people have thus suggested that the use of "duplicate" to describe records that are equivalent when considering only a subset of fields is misleading and doesn't refer to a true duplicate.
We are looking for a suitable term instead of duplicate. Suggestions have included "replicate" but checking various thesauri, this would not appear correct either. Some suggestions are:
TestField | Value |
---|---|
GUID | f2b4a50a-6b2f-4930-b9df-da87b6a21082 |
Label | AMENDMENT_COORDINATES_TRANSPOSED |
Description | Propose amendment of the signs of dwc:decimalLatitude and/or dwc:decimalLongitude to align the location with the dwc:countryCode. |
TestType | Amendment |
Darwin Core Class | Location |
Information Elements ActedUpon | dwc:decimalLatitude |
dwc:decimalLongitude | |
Information Elements Consulted | dwc:countryCode |
Expected Response | INTERNAL_PREREQUISITES_NOT_MET if any of dwc:decimalLatitude or dwc:decimalLongitude or dwc:countryCode are EMPTY; AMENDED dwc:decimalLatitude and dwc:decimalLongitude if the coordinates were transposed or one or more of the signs of the coordinates were reversed to align the location with dwc:countryCode; otherwise NOT_AMENDED |
Data Quality Dimension | Consistency |
Term-Actions | COORDINATES_TRANSPOSED |
Parameter(s) | |
Source Authority | |
Specification Last Updated | 2023-09-17 |
Examples | [dwc:decimalLatitude="25.46", dwc:decimalLongitude="135.87", dwc:countryCode="AU": Response.status=AMENDED, Response.result=dwc:decimalLatitude="-25.46", Response.comment="Input fields contain interpretable values"] |
[dwc:decimalLatitude="25.46", dwc:decimalLongitude="135.87", dwc:countryCode="": Response.status=NOT_AMENDED, Response.result="", Response.comment="dwc:countryCode is EMPTY"] | |
Source | iDigBio, GBIF, BISON, FP, Kurator, ALA |
References |
|
Example Implementations (Mechanisms) | Kurator |
Link to Specification Source Code | https://github.com/FilteredPush/geo_ref_qc/blob/master/src/main/java/org/filteredpush/qc/georeference/DwCGeoRefDQ.java#L324 |
Notes | The dwc:geodeticDatum is not necessary for this test. The maximum positional shift between any geographic coordinate reference system and WGS84 is less than 6 km, so any hemisphere test that relies on a country code for consistency would not be affected by the potential shift. The prior VALIDATION for this test is VALIDATION_COORDINATE_COUNTRYCODE_CONSISTENT (adb27d29-9f0d-4d52-b760-a77ba57a69c9). |
TestField | Value |
---|---|
GUID | 8ab38bee-323c-4926-a7e9-c0417cd3b14d |
Label | AMENDMENT_POLYNOMIAL_STANDARDIZED |
Description | Amend the scientific name to correct typographical errors and misspellings according to a specified source authority. |
TestType | Amendment |
Darwin Core Class | Taxon |
Information Elements ActedUpon | dwc:scientificName |
dwc:genericName | |
dwc:specificEpithet | |
dwc:infraSpecificEpithet | |
dwc:scientificNameAuthorship | |
dwc:yearOfPublication | |
Information Elements Consulted | |
Expected Response | EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if dwc:scientificName is EMPTY; AMENDED (dwc:scientificName, genus, specificEpithet, infraspecificEpithet, scientificNameAuthorship, yearOfPublication) if typographical errors and misspellings represented in dwc:scientificName have been unambiguously interpreted according to the bdq:sourceAuthority; otherwise NOT_CHANGED |
Data Quality Dimension | Conformance |
Term-Actions | POLYNOMIAL_STANDARDIZED |
Parameter(s) | bdq:sourceAuthority |
Source Authority | [bdq:sourceAuthority ](bdq:sourceAuthority default = "GBIF Backbone Taxonomy" {[https://doi.org/10.15468/39omei]} {API endpoint [https://api.gbif.org/v1/species?datasetKey=d7dddbf4-2cf0-4f39-9b2a-bb099caae36c&name=]}) |
Specification Last Updated | 2024-04-16 |
Examples | [dwc:scientificName="Acacia longifloia" Response.status=AMENDED, Response.result=dwc:scientificName"Acacia longifolia" Response.comment="dwc:scientificName contains an interpretable value according to the bdq:sourceAuthority"] |
[dwc:scientificName="Acacia camptophylla": Response.status=NOT_AMENDED, Response.result="", Response.comment="dwc:scientificName does not contain an interpretable value as there are a number of options according to the bdq:sourceAuthority"] | |
Source | Tania Laity |
References |
|
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes | [bdq:sourceAuthority default = GBIF Backbone Taxonomy]. (Currently found at: https://www.gbif.org/en/developer/species). The purpose of this Amendment is to correct errors in spelling and typography only. It is not intended to make changes of a taxonomic nature or to deal with errors or inconsistencies in the format of the Authorship. |
TestField | Value |
---|---|
GUID | ed511370-82bb-48e3-9ab8-6df14845c5c4 |
Label | MEASURE_VALIDATIONTESTS_RUN |
Description | Total number of tests of output type VALIDATION that have been attempted to have been run against the record |
TestType | Measure |
Darwin Core Class | All |
Information Elements ActedUpon | |
Information Elements Consulted | AllDarwinCoreTerms |
Expected Response | INTERNAL_PREREQUISITES_NOT_MET if no tests of type VALIDATION were run; REPORT the number of tests of output type VALIDATION that were run against the record; otherwise NOT_REPORTED |
Data Quality Dimension | Completeness |
Term-Actions | VALIDATIONTESTS_RUN |
Parameter(s) | |
Source Authority | |
Specification Last Updated | 2024-02-22 |
Examples | [ Response.status=RUN_HAS_RESULT, Response.result="101", Response.comment="101 VALIDATION tests have been run"] |
Source | John Wieczorek |
References | |
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes |
TestField | Value |
---|---|
GUID | b9c184ce-a859-410c-9d12-71a338200380 |
Label | VALIDATION_COORDINATES_TERRESTRIALMARINE |
Description | Does the marine/non-marine biome of a taxon from the bdq:sourceAuthority match the biome at the location given by the coordinates? |
TestType | Validation |
Darwin Core Class | Location |
Information Elements ActedUpon | dwc:decimalLatitude |
dwc:decimalLongitude | |
Information Elements Consulted | dwc:scientificName |
Expected Response | EXTERNAL_PREREQUISITES_NOT_MET if either bdq:taxonomyIsMarine or bdq:geospatialLand are not available; INTERNAL_PREREQUISITES_NOT_MET if dwc:dcientificName is EMPTY or the marine/non-marine status of the taxon is not interpretable from bdq:taxonomyIsMarine or the values of dwc:decimalLatitude or dwc:decimalLongitude are EMPTY; COMPLIANT if the taxon marine/non-marine status from bdq:taxonomyIsMarine matches the marine/non-marine status of dwc:decimalLatitude and dwc:decimalLongitude on the boundaries given by bdq:geospatialLand plus an exterior buffer given by bdq:spatialBufferInMeters; otherwise NOT_COMPLIANT |
Data Quality Dimension | Consistency |
Term-Actions | COORDINATES_TERRESTRIALMARINE |
Parameter(s) | bdq:taxonIsMarine |
bdq:geospatialLand | |
bdq:spatialBufferInMeters | |
Source Authority | bdq:taxonIsMarine default = "World Register of Marine Species (WoRMS") {[https://www.marinespecies.org/]} {Web service [https://www.marinespecies.org/aphia.php?p=webservice]} |
{bdq:geospatialLand default = The spatial union of "NaturalEarth 10m-physical-vectors for Land" [https://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/physical/ne_10m_land.zip] and "NaturalEarth Minor Islands" [https://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/physical/ne_10m_minor_islands.zip]} | |
bdq:spatialBufferInMeters default = "3000" | |
Specification Last Updated | 2024-04-15 |
Examples | [dwc:decimalLatitude="-41.0525925872862", dwc:decimalLongitude="-71.5310546742521", dwc:scientificName="Aegla neuquensis": Response.status=RUN_HAS_RESULT, Response.result=COMPLIANT, Response.comment="The species is freshwater aquatic and the coordinates fall in a lake and thus COMPLIANT"] |
[dwc:decimalLatitude="41.0525925872862", dwc:decimalLongitude="-71.5310546742521", dwc:scientificName="Puma concolor": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="dwc:scientificName is terrestrial but coordinates are marine"] | |
Source | ALA, OBIS |
References |
|
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes | dwc:coordinatePrecicision and dwc:coordinateUncertaintyInMeters (if present) imply a potential displacement of the provided coordinates. These two terms can be considered spatial buffers. Likewise, country polygons cannot be 100% accurate at all scales (Dooley 2005), so a spatial buffer of the country boundaries is justified. Taking the spatial buffers into account does however greatly complicate both the logic and the implementation of such tests. The same applies to potential conversion of the Spatial Reference System (SRS) of dwc:decimalLatitude and dwc:decimalLongitude to the SRS used in the bdq:sourceAuthority. |
TestField | Value |
---|---|
GUID | fec5ffe6-3958-4312-82d9-ebcca0efb350 |
Label | AMENDMENT_COUNTRYCODE_STANDARDIZED |
Description | Propose amendment to the value of dwc:countryCode if it can be interpreted as an ISO country code. |
TestType | Amendment |
Darwin Core Class | Location |
Information Elements ActedUpon | dwc:countryCode |
Information Elements Consulted | |
Expected Response | EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISTITES_NOT_MET if the value of dwc:countryCode is EMPTY; AMENDED the value of dwc:countryCode if it can be unambiguously interpreted from bdq:sourceAuthority; otherwise NOT_AMENDED |
Data Quality Dimension | Conformance |
Term-Actions | COUNTRYCODE_STANDARDIZED |
Parameter(s) | |
Source Authority | bdq:sourceAuthority default = "ISO 3166 Country Codes" {[https://www.iso.org/iso-3166-country-codes.html]} {ISO 3166-1-alpha-2 Country Code search [https://www.iso.org/obp/ui/#search]} |
Specification Last Updated | 2023-09-17 |
Examples | [dwc:countryCode="Australia": Response.status=AMENDED, Response.result=dwc:countryCode="AU", Response.comment="dwc:countryCode contains an interpretable value"] |
[dwc:countryCode="Aust.": Response.status=NOT_AMENDED, Response.result=, Response.comment="dwc:countryCode contains an ambiguous value"] | |
Source | |
References |
|
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes |
I learned about the DarwinCore tests list at DarwinCore Hour. Thanks for that tremendous resource! When I visited the Task Group 2 wiki page just now, the link to the spreadsheet worked, but showed me this warning:
File is in owner's trash
You won't be able to access this file soon.
If you still need access, please download it or copy and paste its contents.
before taking me to the file. Should I worry, or is Google perhaps confused by multiple owners?
TestField | Value |
---|---|
GUID | f98a54eb-59e7-44c7-b96f-200e6af1c895 |
Label | ISSUE_DAYMONTH_SWAPPED |
Description | Is it likely that the day and month have been swapped? |
TestType | Validation |
Darwin Core Class | Event |
Information Elements ActedUpon | dwc:day |
dwc:month | |
Information Elements Consulted | |
Expected Response | INTERNAL_PREREQUISITES_NOT_MET if dwc:day or dwc:month are EMPTY; NOT_ISSUE if the value of dwc:day is less than 12 and dwc:month is great than 12; otherwise POTENTIAL_ISSUE |
Data Quality Dimension | Conformance |
Term-Actions | DAYMONTH_SWAPPED |
Parameter(s) | |
Source Authority | |
Specification Last Updated | 2024-02-23 |
Examples | [dwc:day="14, dwc:month="10": Response.status=RUN_HAS_RESULT, Response.result=NOT_ISSUE, Response.comment="day and month are within accepted ranges"] |
[dwc:day="10", dwc:month="14": Response.status=RUN_HAS_RESULT, Response.result=POTENTIAL_ISSUE, Response.comment="dwc:month is not in the accepted range and dwc:day is less than 13"] | |
Source | ALA |
References | |
Example Implementations (Mechanisms) | Kurator |
Link to Specification Source Code | https://github.com/FilteredPush/event_date_qc/blob/master/src/main/java/org/filteredpush/qc/date/DwCEventDQ.java#L723 |
Notes |
TestField | Value |
---|---|
GUID | eaad41c5-1d46-4917-a08b-4fd1d7ff5c0f |
Label | VALIDATION_PHYLUM_FOUND |
Description | Does the value of dwc:phylum occur at rank of Phylum in bdq:sourceAuthority? |
TestType | Validation |
Darwin Core Class | Taxon |
Information Elements ActedUpon | dwc:phylum |
Information Elements Consulted | |
Expected Response | EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if dwc:phylum is EMPTY; COMPLIANT if the value of dwc:phylum was found as a value at the rank of Phylum by the bdq:sourceAuthority; otherwise NOT_COMPLIANT |
Data Quality Dimension | Conformance |
Term-Actions | PHYLUM_FOUND |
Parameter(s) | bdq:sourceAuthority |
Source Authority | bdq:sourceAuthority default = "GBIF Backbone Taxonomy" {[https://doi.org/10.15468/39omei]} {API endpoint [https://api.gbif.org/v1/species?datasetKey=d7dddbf4-2cf0-4f39-9b2a-bb099caae36c&name=]} |
Specification Last Updated | 2022-03-25 |
Examples | [dwc:phylum="Tracheophyta": Response.status=RUN_HAS_RESULT, Response.result=COMPLIANT, Response.comment="dwc:phylum has an equivalent at the rank of Phylum in the bdq:sourceAuthority. GBIF.org uses Trachyophyta for the Phylum including ferns"] |
[dwc:phylum="Trachyophyta": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="dwc:phylum does not have an equivalent at the rank of Phylum in the bdq:sourceAuthority."] | |
Source | iDigBio |
References |
|
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes | The purpose of this test is to check whether the value is a name that is a result of a nomenclatural act at this rank. This excludes unpublished names, misspellings and vernacular names. It is expected that the test will designate the source authority against to check. The same test might return distinct results when using distinct source authorities. |
TestField | Value |
---|---|
GUID | f51e15a6-a67d-4729-9c28-3766299d2985 |
Label | VALIDATION_EVENTDATE_NOTEMPTY |
Description | Is there a value in dwc:eventDate? |
TestType | Validation |
Darwin Core Class | Event |
Information Elements ActedUpon | dwc:eventDate |
Information Elements Consulted | |
Expected Response | COMPLIANT if dwc:eventDate is not EMPTY; otherwise NOT_COMPLIANT |
Data Quality Dimension | Completeness |
Term-Actions | EVENTDATE_NOTEMPTY |
Parameter(s) | |
Source Authority | |
Specification Last Updated | 2023-09-17 |
Examples | [dwc:eventDate="1962-11-01T10:00-0600": Response.status=RUN_HAS_RESULT, Response.result=COMPLIANT, Response.comment="dwc:eventdate is not EMPTY"] |
[dwc:eventDate="": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="dwc:eventDate is EMPTY"] | |
Source | TG2-Gainesville |
References | |
Example Implementations (Mechanisms) | FilteredPush/Kurator:event_date_qc 10.5281/zenodo.596795. |
Link to Specification Source Code | event_date_qc v3.0.0 DwCEventDQ.validationEventdateNotEmpty() |
Notes |
TestField | Value |
---|---|
GUID | 710fe118-17e1-440f-b428-88ba3f547d6d |
Label | AMENDMENT_EVENT_FROM_EVENTDATE |
Description | Propose amendment to values in any of dwc:year, dwc:month, dwc:day, dwc:startDayOfYear or dwc:endDayOfYear from the content of dwc:eventDate. |
TestType | Amendment |
Darwin Core Class | Event |
Information Elements ActedUpon | dwc:year |
dwc:month | |
dwc:day | |
dwc:startDayOfYear | |
dwc:endDayOfYear | |
Information Elements Consulted | dwc:eventDate |
Expected Response | INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1; FILLED_IN (1) dwc:day from dwc:eventDate if dwc:day is EMPTY and dwc:eventDate has a precision of a day or finer and is within a single day, (2) dwc:month from dwc:eventDate if dwc:month is EMPTY and dwc:eventDate has a precision of a single month or finer and is within a single month, (3) dwc:year from dwc:eventDate if dwc:year is EMPTY and dwc:eventDate has a precision of a single year or finer and is within a single year, (4) dwc:startDayOfYear and dwc:endDayOfYear if they are EMPTY and dwc:eventDate has a precision of a day or better; otherwise NOT_AMENDED. |
Data Quality Dimension | Completeness |
Term-Actions | EVENT_FROM_EVENTDATE |
Parameter(s) | |
Source Authority | |
Specification Last Updated | 2023-09-17 |
Examples | [dwc:eventDate="2023-01-26", dwc:year="2023", dwc:month="", dwc:day="", dwc:startDayOfYear="", dwc:endDayOfYear="": Response.status=FILLED_IN, Response.result= dwc:startDayOfYear="26", dwc:month="1", dwc:day="26",dwc:endDayOfYear="26", Response.comment="dwc:month, dwc:day, dwc:startDayOfyear and dwc:endDayOfYear filled in from dwc:eventDate"] |
[dwc:eventDate="2023", dwc:year="2023", dwc:month="", dwc:day="", dwc:startDayOfYear="", dwc:endDayOfYear="": Response.status=NOT_AMENDED, Response.result=, Response.comment="No amendments possible"] | |
Source | VertNet |
References |
|
Example Implementations (Mechanisms) | Kurator:event_date_qc |
Link to Specification Source Code | FilteredPush event_date_qc DwCEventDQ.amendmentEventFromEventdate() unit test in DwcEventDQTest |
Notes | Only fields that are empty will be have changes proposed, and only if dwc:eventDate has a valid ISO 8601-1 date. The dwc:eventDate is the canonical form of the event date (it is the first trusted form). If event date does not contain a range, dwc:startDayOfYear = dwc:endDayOfYear. Time (as compared to date) is not deemed a CORE component. Note, see sequencing tests section of standards document, run this amendment after any other amendment which may affect dwc:eventDate |
Write introduction to Task Group 2 in the Read Me file of the Interest Group (https://tdwg.github.io/bdq/tg2/)
TestField | Value |
---|---|
GUID | 3cff4dc4-72e9-4abe-9bf3-8a30f1618432 |
Label | VALIDATION_EVENTDATE_INRANGE |
Description | Is the value of dwc:eventDate entirely with the Parameter Range? |
TestType | Validation |
Darwin Core Class | Event |
Information Elements ActedUpon | dwc:eventDate |
Information Elements Consulted | |
Expected Response | INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or if the value of dwc:eventDate is not a valid ISO 8601-1 date; COMPLIANT if the range of dwc:eventDate is entirely within the range bdq:earliestValidDate to bdq:latestValidDate, inclusive, otherwise NOT_COMPLIANT |
Data Quality Dimension | Conformance |
Term-Actions | EVENTDATE_INRANGE |
Parameter(s) | bdq:earliestValidDate |
bdq:latestValidDate | |
Source Authority | bdq:earliestValidDate default ="1582-11-15" |
bdq:latestValidDate default = current year | |
Specification Last Updated | 2023-09-17 |
Examples | [dwc:eventDate="1962-11-01T10:00-0600": Response.status=RUN_HAS_RESULT, Response.result=COMPLIANT, Response.comment="dwc:eventDate is IN_RANGE"] |
[dwc:eventDate="2300-11-01T10:00": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="dwc:eventDate is NOT_IN_RANGE"] | |
Source | VertNet |
References |
|
Example Implementations (Mechanisms) | Kurator:event_date_qc |
Link to Specification Source Code | FilteredPush event_date_qc DwCEventDQ.validationEventdateInrange() |
Notes | This test provides for a default earliest date, which is 1582-11-15 by convention. That date was chosen because ISO 8601-1 asserts that "the use of proleptic Gregorian calendar dates prior are not allowed in ISO 8601-1 without prior agreement of the parties exchanging data", and Darwin Core does not comment on this. Different calendars have been used at different times in different places, and the transcription of an original date in one calendar into dwc:eventDate, where a Gregorian Calendar is assumed, may or may not have been done with the correct translation of the date, and metadata may or not be present to even identify such records. Given the complexity, and ongoing nature of transitions between calendars, we do not advocate using this test for quality assurance by selecting a transition date and using it as a threshold. |
TestField | Value |
---|---|
GUID | 453844ae-9df4-439f-8e24-c52498eca84a |
Label | MEASURE_VALIDATIONTESTS_NOTCOMPLIANT |
Description | The number of distinct VALIDATION tests that have a Response.status="NOT_COMPLIANT" for a given record. |
TestType | Measure |
Darwin Core Class | All |
Information Elements ActedUpon | |
Information Elements Consulted | AllDarwinCoreTerms |
Expected Response | INTERNAL_PREREQUISITES_NOT_MET if no tests of type VALIDATION were attempted to be run; REPORT of the number of tests of output type VALIDATION run against the record that have Response.result="NOT_COMPLIANT; otherwise NOT_REPORTED |
Data Quality Dimension | Reliability |
Term-Actions | VALIDATIONTESTS_NOTCOMPLIANT |
Parameter(s) | |
Source Authority | |
Specification Last Updated | 2023-09-17 |
Examples | [Response.status=RUN_HAS_RESULT, Response.result="37", Response.comment="37 VALIDATION tests had Response.status="NOT_COMPLIANT."] |
Source | Lee Belbin |
References | |
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes | We have three individual measures for pass (MEASURE_VALIDATIONTESTS_COMPLIANT (45fb49eb-4a1b-4b49-876f-15d5034dfc73)), fail (MEASURE_VALIDATIONTESTS_NOTCOMPLIANT (453844ae-9df4-439f-8e24-c52498eca84a)), and PREREQUISITES_NOT_MET (49a94636-a562-4e6b-803c-665c80628a3d). To get the total number of tests that were attempted, add all three measures. To get the total number of tests that ran, add NOT_COMPLIANT (fail) and COMPLIANT (pass). |
TestField | Value |
---|---|
GUID | 58486cb6-1114-4a8a-ba1e-bd89cfe887e9 |
Label | VALIDATION_LOCATION_NOTEMPTY |
Description | Is there a value in any of the Darwin Core spatial terms that could specify a location? |
TestType | Validation |
Darwin Core Class | Location |
Information Elements ActedUpon | dwc:higherGeographyID |
dwc:higherGeography | |
dwc:continent | |
dwc:country | |
dwc:countryCode | |
dwc:stateProvince | |
dwc:county | |
dwc:municipality | |
dwc:waterBody | |
dwc:island | |
dwc:islandGroup | |
dwc:locality | |
dwc:locationID | |
dwc:verbatimLocality | |
dwc:decimalLatitude | |
dwc:decimalLongitude | |
dwc:verbatimCoordinates | |
dwc:verbatimLatitude | |
dwc:verbatimLongitude | |
dwc:footprintWKT | |
Information Elements Consulted | |
Expected Response | COMPLIANT if at least one term needed to determine the location of the entity exists and is not EMPTY; otherwise NOT_COMPLIANT |
Data Quality Dimension | Completeness |
Term-Actions | LOCATION_NOTEMPTY |
Parameter(s) | |
Source Authority | |
Specification Last Updated | 2023-09-18 |
Examples | [dwc:locationID="https://opencontext.org/subjects/768A875F-E205-4D0B-DE55-BAB7598D0FD19", dwc:higherGeographyID="", dwc:higherGeography="", dwc:continent="", dwc:waterBody="", dwc:islandGroup="", dwc:island="", dwc:country="", dwc:countryCode="", dwc:stateProvince="", dwc:county="", dwc:municipality="", dwc:locality="", dwc:verbatimLocality="", dwc:decimalLatitude="", dwc:decimalLongitude="", dwc:coordinateUncertaintyInMeters="", dwc:geodeticDatum="", dwc:verbatimCoordinates="", dwc:verbatimLatitude="", dwc:verbatimLongitude="": Response.status=RUN_HAS_RESULT, Response.result=COMPLIANT, Response.comment="Location not EMPTY as there is a value in dwc:locationID"] |
[dwc:locationID="", dwc:higherGeographyID="", dwc:higherGeography="", dwc:continent="", dwc:waterBody="", dwc:islandGroup="", dwc:island="", dwc:country="", dwc:countryCode="", dwc:stateProvince="", dwc:county="", dwc:municipality="", dwc:locality="", dwc:verbatimLocality="", dwc:decimalLatitude="", dwc:decimalLongitude="", dwc:coordinateUncertaintyInMeters="", dwc:geodeticDatum="", dwc:verbatimCoordinates="", dwc:verbatimLatitude="", dwc:verbatimLongitude="": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="All location fields are EMPTY."] | |
Source | Lee Belbin |
References |
|
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes | Only fails if all of the relevant fields of the Darwin Core Location class are EMPTY or do not exist. Relevant Darwin Core fields include dwc:locationID, dwc:higherGeographyID, dwc:higherGeography, dwc:continent, dwc:waterBody, dwc:islandGroup, dwc:island, dwc:country, dwc:countryCode, dwc:stateProvince, dwc:county, dwc:municipality, dwc:locality, dwc:verbatimLocality, dwc:decimalLatitude, dwc:decimalLongitude, dwc:verbatimCoordinates, dwc:verbatimLatitude, dwc:verbatimLongitude, dwc:footprintWKT. Elevation and/or depth alone are deemed insufficient to meaningfully locate a position on the earth. |
I have uploaded a summary of the data quality meetings held during the week of TDWG2017 in Ottawa. The main document summarises the work done on the Sunday. At the end is a link to a document (on Google Docs) of the Workshop held on Thursday afternoon.
The document can be found in the Wiki
If we have made an error in capturing what was discussed at the Workshop, then please make a comment in the Google Doc.
I thank all of you who participate in Ottawa. It was a very successful series of meetings. Also a special thanks to all the presenters at the data Quality Symposia.
We are always looking for more people to help with the work of the Task groups, so please do not hesitate to get involved.
-Lee Belbin, who has worked with TDWG, GBIF and the Atlas of Living Australia (ALA) is leading this Task Group. We have 33 people who have expressed an interest in this Task Group so far, so now is the time for you to have your say and to contribute to this Task Group. More information to follow shortly.
-Arthur
TestField | Value |
---|---|
GUID | 69b2efdc-6269-45a4-aecb-4cb99c2ae134 |
Label | VALIDATION_COUNTRY_FOUND |
Description | Does the value of dwc:country occur in bdq:sourceAuthority? |
TestType | Validation |
Darwin Core Class | Location |
Information Elements ActedUpon | dwc:country |
Information Elements Consulted | |
Expected Response | EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if dwc:country is EMPTY; COMPLIANT if value of dwc:country is a place type equivalent to "nation" by the bdq:sourceAuthority; otherwise NOT_COMPLIANT |
Data Quality Dimension | Conformance |
Term-Actions | COUNTRY_FOUND |
Parameter(s) | bdq:sourceAuthority |
Source Authority | bdq:sourceAuthority default = "The Getty Thesaurus of Geographic Names (TGN)" {[https://www.getty.edu/research/tools/vocabularies/tgn/index.html]} |
Specification Last Updated | 2024-04-15 |
Examples | [dwc:country="Eswatini": Response.status=RUN_HAS_RESULT, Response.result=COMPLIANT, Response.comment="dwc:country is a valid country name according to The Getty Thesaurus of Geographic Names (2021-03-30)."] |
[dwc:country="Swaziland": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="Eswatini is the preferred name according to The Getty Thesaurus of Geographic Names (2021-03-30)."] | |
Source | ALA, GBIF |
References |
|
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes | Non-country information such "high seas" will fail this test. Multiple values in the dwc:country field (whether to signify on a border or in a list of possibilities) will fail this test. Locations outside of a jurisdiction covered by a country code should not have a value in the field dwc:countryCode. This test must return NOT_COMPLIANT if there is leading or trailing whitespace or there are leading or trailing non-printing characters. |
TestField | Value |
---|---|
GUID | 0949110d-c06b-450e-9649-7c1374d940d1 |
Label | VALIDATION_DECIMALLONGITUDE_INRANGE |
Description | Is the value of dwc:decimalLongitude a number between -180 and 180 inclusive? |
TestType | Validation |
Darwin Core Class | Location |
Information Elements ActedUpon | dwc:decimalLongitude |
Information Elements Consulted | |
Expected Response | INTERNAL_PREREQUISITES_NOT_MET if dwc:decimalLongitude is EMPTY or the value is not a number; COMPLIANT if the value of dwc:decimalLongitude is between -180 and 180 degrees, inclusive; otherwise NOT_COMPLIANT |
Data Quality Dimension | Conformance |
Term-Actions | DECIMALLONGITUDE_INRANGE |
Parameter(s) | |
Source Authority | |
Specification Last Updated | 2023-09-17 |
Examples | [dwc:decimalLongitude="0": Response.status=RUN_HAS_RESULT, Response.result=COMPLIANT, Response.comment="dwc:decimalLongitude is in range"] |
[dwc:decimalLongitude="181.0554": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="dwc:decimalLongitude >180"] | |
Source | ALA, GBIF, OBIS |
References |
|
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes |
TestField | Value |
---|---|
GUID | 63c3da43-7935-4265-8fe7-013013c60d06 |
Label | AMENDMENT_DAY_MONTH_TRANSPOSED |
Description | Swap dwc:month and dwc:day if dwc:month is greater than 12 and dwc:day is less than 12. |
TestType | Amendment |
Darwin Core Class | Event |
Information Elements ActedUpon | dwc:day |
dwc:month | |
Information Elements Consulted | |
Expected Response | AMENDED the values of dwc:month and dwc:day if dwc:month is greater than 12 and dwc:day is less than 12; otherwise NOT_AMENDED |
Data Quality Dimension | Conformance |
Term-Actions | DAY_MONTH_TRANSPOSED |
Parameter(s) | |
Source Authority | |
Specification Last Updated | 2024-02-22 |
Examples | [dwc:month="14", dwc:day="2": Response.status=AMENDED, Response.Result=dwc:month="2", dwc:day="14", Response.comment="month was greater than 12 and day less than 12 so transposed values"] |
[dwc:month="2", dwc:day="14": Response.status=NOT_AMENDED, Response.Result=dwc:month="2", dwc:day="14", Response.comment="month was less than 12 and day was greater than 12 so no amendment applies"] | |
Source | ALA |
References | |
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes |
TestField | Value |
---|---|
GUID | 20eeeaf8-0d34-47da-a1a5-e9b6df4f3ace |
Label | AMENDMENT_CLASS_STANDARDIZED |
Description | Can the value of dwc:class be standardized using the Source Authority? |
TestType | Amendment |
Darwin Core Class | Taxon |
Information Elements ActedUpon | dwc:class |
Information Elements Consulted | |
Expected Response | EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if dwc:class is EMPTY; AMENDED dwc:class if it can be been unambiguously interpreted according to the bdq:sourceAuthority; otherwise NOT_CHANGED |
Data Quality Dimension | Conformance |
Term-Actions | CLASS_STANDARDIZED |
Parameter(s) | bdq:sourceAuthority |
Source Authority | bdq:sourceAuthority default = "GBIF Backbone Taxonomy" {[https://doi.org/10.15468/39omei]} {API endpoint [https://api.gbif.org/v1/species?datasetKey=d7dddbf4-2cf0-4f39-9b2a-bb099caae36c&name=]} |
Specification Last Updated | 2024-04-16 |
Examples | [dwc:class="Birds": Response.status=AMENDED, Response.result=dwc:class="Aves", Response.comment="Input field contains interpretable value"] |
[dwc:class="X": Response.status=NOT_AMENDED, Response.result="", Response.comment="order not found in source authority"] | |
Source | iDigBio |
References | |
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes |
TestField | Value |
---|---|
GUID | 3667556d-d8f5-454c-922b-af8af38f613c |
Label | VALIDATION_FAMILY_FOUND |
Description | Does the value of dwc:family occur at rank of Family in bdq:sourceAuthority? |
TestType | Validation |
Darwin Core Class | Taxon |
Information Elements ActedUpon | dwc:family |
Information Elements Consulted | |
Expected Response | EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if dwc:family is EMPTY; COMPLIANT if the value of dwc:family was found as a value at the rank of Family by the bdq:sourceAuthority; otherwise NOT_COMPLIANT |
Data Quality Dimension | Conformance |
Term-Actions | FAMILY_FOUND |
Parameter(s) | bdq:sourceAuthority |
Source Authority | bdq:sourceAuthority default = "GBIF Backbone Taxonomy" {[https://doi.org/10.15468/39omei]} {API endpoint [https://api.gbif.org/v1/species?datasetKey=d7dddbf4-2cf0-4f39-9b2a-bb099caae36c&name=]} |
Specification Last Updated | 2023-09-17 |
Examples | [dwc:family="Agaricaceae": Response.status=RUN_HAS_RESULT, Response.result=COMPLIANT, Response.comment="bdq:family has an equivalent at the rank of Family in the bdq:sourceAuthority"] |
[dwc:family="Agaricacae": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="bdq:family does not have an equivalent at the rank of Family in the Parameterized Source Authority"] | |
Source | iDigBio |
References |
|
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes | The purpose of this test is to check whether the value is a name that is a result of a nomenclatural act at this rank. This excludes unpublished names, misspellings and vernacular names. It is expected that the test will designate the source authority against to check. The same test might return distinct results when using distinct source authorities. |
TestField | Value |
---|---|
GUID | 3cfe9ab4-79f8-4afd-8da5-723183ef16a3 |
Label | VALIDATION_OCCURRENCEID_STANDARD |
Description | Does dwc:occurrenceID contain a valid identifier? |
TestType | Validation |
Darwin Core Class | Occurrence |
Information Elements ActedUpon | dwc:occurrenceID |
Information Elements Consulted | |
Expected Response | INTERNAL_PREREQUISITES_NOT_MET if dwc:ocurrenceID is EMPTY; COMPLIANT if (1) dwc:occurrenceID is a validly formed LSID, or (2) dwc:occurrenceID is a validly formed URN with at least NID and NSS present, or (3) dwc:occurrenceID is in the form scope:value, or (4) dwc:occurrenceID is a validly formed URI with host and path where path consists of more than just "/"; otherwise NOT_COMPLIANT. |
Data Quality Dimension | Conformance |
Term-Actions | OCCURRENCEID_COMPLETE |
Parameter(s) | |
Source Authority | |
Specification Last Updated | 2024-04-02 |
Examples | [dwc:occurrenceID="https://www.inaturalist.org/observations/43047701": Response.status=RUN_HAS_RESULT, Response.result=COMPLIANT, Response.comment="dwc:occurrenceID conforms to GUID structure"] |
[dwc:occurrenceID="42": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="dwc:occurrenceID does not conform to GUID structure"] | |
Source | VertNet |
References |
|
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes |
TestField | Value |
---|---|
GUID | 3136236e-04b6-49ea-8b34-a65f25e3aba1 |
Label | VALIDATION_LICENSE_STANDARD |
Description | Does the value of dcterms:license occur in bdq:sourceAuthority? |
TestType | Validation |
Darwin Core Class | Record-level |
Information Elements ActedUpon | dcterms:license |
Information Elements Consulted | |
Expected Response | EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if dcterms:license is EMPTY; COMPLIANT if the value of the term dcterms:license is in the bdq:sourceAuthority; otherwise NOT_COMPLIANT |
Data Quality Dimension | Conformance |
Term-Actions | LICENSE_STANDARD |
Parameter(s) | bdq:sourceAuthority |
Source Authority | bdq:sourceAuthority default = "Creative Commons 4.0 Licenses or CC0 {[https://creativecommons.org/]} { Regular Expression [((http(s){0,1}://creativecommons.org/licenses/(by|by-sa|by-nc|by-nc-sa|by-nd|by-nc-nd)/4.0/((deed|legalcode)(.(id|eu|da|de|en|es|fr|fy|hr|it|lv|lt|mi|ni|no|pl|pt|ro|si|fi|sv|tr|cs|el|ru|uk|ar|jp|zh-hans|zh-hant|ko)){0,1})|(http(s){0,1}://creativecommons.org/publicdomain/zero/1.0/((deed|legalcode)(.(id|eu|da|de|en|es|fr|fy|hr|it|lv|lt|ni|no|pl|pt|ro|si|fi|sv|tr|cs|el|ru|uk|ar|jp|zh-hans|zh-hant|ko)){0,1})))$ |
Specification Last Updated | 2023-09-17 |
Examples | [dcterms:license="CC BY": Response.status=RUN_HAS_RESULT, Response.result=COMPLIANT, Response.comment="dcterms:license matches a term in bdq:sourceAuthority"] |
[dcterms:license="GPL": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="dcterms:license does not match a term in the bdq:sourceAuthority"] | |
Source | John Wieczorek |
References |
|
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes | The license at the record level might be derived from the license of the data set from which the record is retrieved. This test must return NOT_COMPLIANT if there is leading or trailing whitespace or there are leading or trailing non-printing characters. The canonical form of the Creative Commons license IRI has nothing after the version e.g. https://creativecommons.org/licenses/by/4.0/, but may be followed by deed or legalcode e.g. https://creativecommons.org/licenses/by/4.0/deed and this may be followed by a language code. However, only some two letter language codes have translations, and some translations are identified by a longer string than the two letter language code. Errors in the language code, or specifying a language code for which a translation doesn't exist returns a 404 error instead of redirecting to the more general license IRI. As of 2024-02-28 deed.mi doesn't exist yet, but legalcode.mi does. |
Write introduction to TG1 in the Read Me file fr the Task Group (https://tdwg.github.io/bdq/tg1/)
TestField | Value |
---|---|
GUID | 0493bcfb-652e-4d17-815b-b0cce0742fbe |
Label | VALIDATION_COUNTRYCODE_STANDARD |
Description | Is the value of dwc:countryCode a valid ISO 3166-1-alpha-2 country code? |
TestType | Validation |
Darwin Core Class | Location |
Information Elements ActedUpon | dwc:countryCode |
Information Elements Consulted | |
Expected Response | EXTERNAL_PREREQUISITES_NOT_MET if bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if the dwc:countryCode is EMPTY; COMPLIANT if dwc:countryCode can be unambiguously interpreted as a valid ISO 3166-1-alpha-2 country code; otherwise NOT_COMPLIANT |
Data Quality Dimension | Conformance |
Term-Actions | COUNTRYCODE_STANDARD |
Parameter(s) | |
Source Authority | bdq:sourceAuthority default = "ISO 3166 Country Codes" {[https://www.iso.org/iso-3166-country-codes.html]} {ISO 3166-1-alpha-2 Country Code search [https://www.iso.org/obp/ui/#search]} |
Specification Last Updated | 2024-04-15 |
Examples | [dwc:countryCode="GL": Response.status=RUN_HAS_RESULT, Response.result=COMPLIANT, Result.comment=" dwc:countryCode is a valid ISO (ISO 3166-1-alpha-2 country codes) value"] |
[dwc:countryCode="GRL": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="dwc:countryCode is NOT a valid ISO (ISO 3166-1-alpha-2 country codes) value"] | |
Source | TG2 |
References |
|
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes | Locations outside of a jurisdiction covered by a country code should not have a value in the field dwc:countryCode. This test must return NOT_COMPLIANT if there is leading or trailing whitespace or there are leading or trailing non-printing characters. |
TestField | Value |
---|---|
GUID | c486546c-e6e5-48a7-b286-eba7f5ca56c4 |
Label | VALIDATION_OCCURRENCEID_NOTEMPTY |
Description | Is there a value in dwc:occurrenceID? |
TestType | Validation |
Darwin Core Class | Occurrence |
Information Elements ActedUpon | dwc:occurrenceID |
Information Elements Consulted | |
Expected Response | COMPLIANT if dwc:occurrenceID is not EMPTY; otherwise NOT_COMPLIANT |
Data Quality Dimension | Completeness |
Term-Actions | OCCURRENCEID_NOTEMPTY |
Parameter(s) | |
Source Authority | |
Specification Last Updated | 2023-09-17 |
Examples | [dwc:occurrenceID="https://www.inaturalist.org/observations/43047701": Response.status=RUN_HAS_RESULT, Response.result=COMPLIANT, Response.comment="dwc:occurrenceID conforms to GUID structure"] |
[dwc:occurrenceID="": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="dwc:occurrenceID is EMPTY"] | |
Source | VertNet |
References | |
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes |
TestField | Value |
---|---|
GUID | c5658b83-4471-4f57-9d94-bf7d0a96900c |
Label | AMENDMENT_MINDEPTH-MAXDEPTH_FROM_VERBATIM |
Description | Propose amendments of the values of dwc:minimumDepthInMeters and/or dwc:maximumDepthInMeters if they can be interpreted from dwc:verbatimDepth. |
TestType | Amendment |
Darwin Core Class | Location |
Information Elements ActedUpon | dwc:minimumDepthInMeters |
dwc:maximumDepthInMeters | |
Information Elements Consulted | dwc:verbatimDepth |
Expected Response | INTERNAL_PREREQUISITES_NOT_MET if dwc:verbatimDepth is EMPTY or the value is not unambiguously interpretable or dwc:minimumDepthInMeters and dwc:maximumDepthInMeters are not EMPTY; FILLED_IN the value of dwc:minimumDepthInMeters and/or dwc:maximumDepthInMeters if they could be unambiguously determined from dwc:verbatimDepth; otherwise NOT_AMENDED |
Data Quality Dimension | Completeness |
Term-Actions | MINDEPTH-MAXDEPTH_FROM_VERBATIM |
Parameter(s) | |
Source Authority | |
Specification Last Updated | 2023-09-17 |
Examples | [dwc:minimumDepthInMeters="", dwc:maximumDepthInMeters="11", dwc:verbatimDepth="Min depth 10m": Response.status=FILLED_IN, Response.result=dwc:minimumDepthInMeters="10", Response.comment="dwc:verbatimDepth contains an interpretable value"] |
[dwc:minimumDepthInMeters="", dwc:maximumDepthInMeters="11", dwc:verbatimDepth="Max depth 100m": Response.status=NOT_AMENDED, Response.result="", Response.comment="Input and output fields are conflicting"] | |
Source | |
References |
|
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes | If the dwc:verbatimDepth has a single value rather than a range, the minimum and maximum values should be amended with the same value. |
TestField | Value |
---|---|
GUID | c09ecbf9-34e3-4f3e-b74a-8796af15e59f |
Label | VALIDATION_YEAR_NOTEMPTY |
Description | Is there a value in dwc:year? |
TestType | Validation |
Darwin Core Class | Event |
Information Elements ActedUpon | dwc:year |
Information Elements Consulted | |
Expected Response | COMPLIANT if dwc:year is not EMPTY; otherwise NOT_COMPLIANT |
Data Quality Dimension | Completeness |
Term-Actions | YEAR_NOTEMPTY |
Parameter(s) | |
Source Authority | |
Specification Last Updated | 2023-09-17 |
Examples | [dwc:year="1949": Response.status=RUN_HAS_RESULT, Response.result=COMPLIANT, Response.comment="dwc:year is not EMPTY"] |
[dwc:year="": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="dwc:year is EMPTY"] | |
Source | TG2-Gainesville |
References | |
Example Implementations (Mechanisms) | FilteredPush/Kurator:event_date_qc 10.5281/zenodo.596795. |
Link to Specification Source Code | event_date_qc v3.0.0 DwCEventDQ.validationYearNotEmpty() |
Notes |
TestField | Value |
---|---|
GUID | f18a470b-3fe1-4aae-9c65-a6d3db6b550c |
Label | VALIDATION_COORDINATES_STATEPROVINCE_CONSISTENT |
Description | Do the geographic coordinates fall on or within the boundary from the bdq:sourceAuthority for the given dwc:stateProvince or within the distance given by bdq:spatialBufferInMeters outside that boundary? |
TestType | Validation |
Darwin Core Class | Location |
Information Elements ActedUpon | dwc:stateProvince |
dwc:decimalLatitude | |
dwc:decimalLongitude | |
Information Elements Consulted | |
Expected Response | EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if the values of dwc:decimalLatitude, dwc:decimalLongitude, or dwc:stateProvince are EMPTY or invalid; COMPLIANT if the geographic coordinates fall on or within the boundary from the bdq:sourceAuthority for the given dwc:stateProvince (after coordinate reference system transformations, if any, have been accounted for), or within the distance given by bdq:spatialBufferInMeters outside that boundary; otherwise NOT_COMPLIANT. |
Data Quality Dimension | Consistency |
Term-Actions | COORDINATES_STATEPROVINCE_CONSISTENT |
Parameter(s) | bdq:sourceAuthority |
bdq:spatialBufferInMeters | |
Source Authority | bdq:sourceAuthority default = "ADM1 boundaries" {[https://gadm.org]} |
bdq:spatialBufferInMeters default = "3000" | |
Specification Last Updated | 2024-04-16 |
Examples | [dwc:stateProvince="Rio Negro", dwc:decimalLatitude="-41.0525925872862", dwc:decimalLongitude="-71.5310546742521", dwc:geodeticDatum="": Response.status=RUN_HAS_RESULT, Response.result=COMPLIANT, Response.comment="Input fields contain interpretable values"] |
[dwc:stateProvince="Neuquén", dwc:decimalLatitude="-41.0525925872862", dwc:decimalLongitude="-71.5310546742521", dwc:geodeticDatum="": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="Input fields contain interpretable values but coordinates don't match dwc:stateProvince"] | |
[dwc:stateProvince="Neuquén", dwc:decimalLatitude="-141.0525925872862", dwc:decimalLongitude="-71.5310546742521", dwc:geodeticDatum="": Response.status=INTERNAL_PREREQUISTES_NOT_MET, Response.comment="Input field contain invalid value"] | |
Source | ALA |
References |
|
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes | The geographic determination service is expected to return a list of names of first-level administrative divisions for geometries that the geographic point falls on or within, including a 3 km buffer around the administrative geometry. A match on any of those names should constitute a consistency, and dwc:countryCode should not be needed to make this determination, that is, this test does not attempt to disambiguate potential duplicate first-level administrative division names. The level of buffering may be related to the scale of the underlying GIS layer being used. At a global scale, typical map scales used for borders and coastal areas are either 1:3M or 1:1M (Dooley 2005, Chapter 4). Horizontal accuracy at those scales is around 1.5-2.5km and 0.5-0.85 km respectively (Chapman & Wieczorek 2020). |
TestField | Value |
---|---|
GUID | 8f1e6e58-544b-4365-a569-fb781341644e |
Label | VALIDATION_MINDEPTH_LESSTHAN_MAXDEPTH |
Description | Is the value of dwc:minimumDepthInMeters a number that is less than or equal to the value of dwc:maximumDepthInMeters? |
TestType | Validation |
Darwin Core Class | Location |
Information Elements ActedUpon | dwc:minimumDepthInMeters |
dwc:maximumDepthInMeters | |
Information Elements Consulted | |
Expected Response | INTERNAL_PREREQUISITES_NOT_MET if dwc:minimumDepthInMeters or dwc:maximumDepthInMeters is EMPTY, or if either are interpretable as not zero or a positive number; COMPLIANT if the value of dwc:minimumDepthInMeters is less than or equal to the value of dwc:maximumDepthInMeters; otherwise NOT_COMPLIANT |
Data Quality Dimension | Conformance |
Term-Actions | MINDEPTH_LESSTHAN_MAXDEPTH |
Parameter(s) | |
Source Authority | |
Specification Last Updated | 2023-09-18 |
Examples | [dwc:minimumDepthInMeters="0", dwc:maximumDepthInMeters="0": Response.status=RUN_HAS_RESULT, Response.result=COMPLIANT, Response.comment="dwc:minimumDepthInMeters = dwc:maximumDepthInMeters"] |
[dwc:minimumDepthInMeters="1", dwc:maximumDepthInMeters="0": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="dwc:minimumDepthInMeters > dwc:maximumDepthInMeters"] | |
Source | ALA, GBIF, OBIS |
References |
|
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes |
TestField | Value |
---|---|
GUID | 0c8dca4c-ecd0-4ba5-bc81-9f8cbea27374 |
Label | AMENDMENT_ORDER_STANDARDIZED |
Description | Can the value of dwc:order be standardized using the Source Authority? |
TestType | Amendment |
Darwin Core Class | Taxon |
Information Elements ActedUpon | dwc:order |
Information Elements Consulted | |
Expected Response | EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if dwc:order is EMPTY; AMENDED dwc:order if it can be been unambiguously interpreted according to the bdq:sourceAuthority; otherwise NOT_CHANGED |
Data Quality Dimension | Conformance |
Term-Actions | ORDER_STANDARDIZED |
Parameter(s) | bdq:sourceAuthority |
Source Authority | bdq:sourceAuthority default = "GBIF Backbone Taxonomy" {[https://doi.org/10.15468/39omei]} {API endpoint [https://api.gbif.org/v1/species?datasetKey=d7dddbf4-2cf0-4f39-9b2a-bb099caae36c&name=]} |
Specification Last Updated | 2024-04-16 |
Examples | [dwc:order="Butterflies": Response.status=AMENDED, Response.result=dwc:order="Lepitoptera", Response.comment="Input field contains interpretable value"] |
[dwc:order="X": Response.status=NOT_AMENDED, Response.result="", Response.comment="order not found in source authority"] | |
Source | iDigBio |
References | |
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes |
TestField | Value |
---|---|
GUID | 28ce32c6-fae1-478b-bf54-3f0b688f37be |
Label | AMENDMENT_FAMILY_STANDARDIZED |
Description | Can the value of dwc:family be standardized using the Source Authority? |
TestType | Amendment |
Darwin Core Class | Taxon |
Information Elements ActedUpon | dwc:family |
Information Elements Consulted | |
Expected Response | EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if dwc:family is EMPTY; AMENDED dwc:family if it can be been unambiguously interpreted according to the bdq:sourceAuthority; otherwise NOT_CHANGED. |
Data Quality Dimension | Conformance |
Term-Actions | FAMILY_STANDARDIZED |
Parameter(s) | bdq:sourceAuthority |
Source Authority | bdq:sourceAuthority default = "GBIF Backbone Taxonomy" {[https://doi.org/10.15468/39omei]} {API endpoint [https://api.gbif.org/v1/species?datasetKey=d7dddbf4-2cf0-4f39-9b2a-bb099caae36c&name=]} |
Specification Last Updated | 2024-04-16 |
Examples | [dwc:family="Kangaroos": Response.status=AMENDED, Response.result=dwc:family="Macropodidae", Response.comment="Input field contains interpretable value"] |
[dwc:family="X": Response.status=NOT_AMENDED, Response.result="", Response.comment="family not found in source authority"] | |
Source | |
References | |
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes |
TestField | Value |
---|---|
GUID | fecaa8a3-bbd8-4c5a-a424-13c37c4bb7b1 |
Label | ISSUE_ANNOTATION_NOTEMPTY |
Description | Are there any annotations associated with the record? |
TestType | Issue |
Darwin Core Class | All |
Information Elements ActedUpon | |
Information Elements Consulted | AllDarwinCoreTerms |
Expected Response | EXTERNAL_PREREQUISITES_NOT_MET if bdq:annotationSystem is not available; POTENTIAL_ISSUE if an annotation in bdq:annotationSystem exists with a matching bdq:annotationAlertIf; otherwise NOT_ISSUE. |
Data Quality Dimension | Reliability |
Term-Actions | ANNOTATION_NOTEMPTY |
Parameter(s) | bdq:annotationSystem |
bdq:annotationAlertIf | |
Source Authority | bdq:annotationSystem default = "W3C Web Annotation" {[https://www.w3.org/annotation/]} {"oa:Annotation vocabulary" {[https://www.w3.org/TR/annotation-vocab/]} |
bdq:annotationAlertIf default = "oa:Annotation with oa:hasTarget having as object any dwciri:term instance that is part of the SingleRecord under test." {[https://www.w3.org/TR/annotation-vocab/]} | |
Specification Last Updated | 2023-09-17 |
Examples | [bdq:annotationAlertIf="": Response.status=RUN_HAS_RESULT, Response.result=NOT_ISSUE, Response.comment="bdq:annotationAlertIf is EMPTY"] |
[bdq:annotationAlertIf="?": Response.status=RUN_HAS_RESULT, Response.result=POTENTIAL_ISSUE, Response.comment="bdq:annotationAlertIf is not EMPTY"] | |
Source | ALA, Lee Belbin |
References | W3C Web annotation Data Model: https://www.w3.org/TR/annotation-model/ |
W3C Web Annotation Vocabulary: https://www.w3.org/TR/annotation-vocab/#annotation | |
TDWG Annotations Interest Group [https://www.tdwg.org/community/annotations/] | |
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes | While there is a W3C standard on 'web annotation', there is no TDWG recommendation on how this standard could be applied to annotating Darwin Core records. While implementation of this test is currently problematic, TG2 considers annotations attached to any aspect of a Darwin Core record justifies this test as a placeholder in the hope of future developments. |
TestField | Value |
---|---|
GUID | adb27d29-9f0d-4d52-b760-a77ba57a69c9 |
Label | VALIDATION_COORDINATES_COUNTRYCODE_CONSISTENT |
Description | Do the geographic coordinates fall on or within the boundaries of the territory given in dwc:countryCode or its Exclusive Economic Zone? |
TestType | Validation |
Darwin Core Class | Location |
Information Elements ActedUpon | dwc:countryCode |
dwc:decimalLatitude | |
dwc:decimalLongitude | |
Information Elements Consulted | |
Expected Response | EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if one or more of dwc:decimalLatitude, dwc:decimalLongitude, or dwc:countryCode are EMPTY or invalid; COMPLIANT if the geographic coordinates fall on or within the boundary defined by the union of the boundary of the country from dwc:countryCode plus it's Exclusive Economic Zone as found in the bdq:sourceAuthority, if any, plus an exterior buffer given by bdq:spatialBufferInMeters; otherwise NOT_COMPLIANT |
Data Quality Dimension | Consistency |
Term-Actions | COORDINATES_COUNTRYCODE_CONSISTENT |
Parameter(s) | bdq:sourceAuthority |
bdq:spatialBufferInMeters | |
Source Authority | bdq:sourceAuthority default = "ADM1 boundaries UNION with Exclusive Economic Zones" {[https://gadm.org] spatial UNION [https://marineregions.org]} |
bdq:spatialBufferInMeters default = "3000" | |
Specification Last Updated | 2023-09-17 |
Examples | [dwc:countryCode="AR", dwc:decimalLatitude="-41.0525925872862", dwc:decimalLongitude="-71.5310546742521": Response.status=RUN_HAS_RESULT, Response.result=COMPLIANT, Response.comment="Coordinates match dwc:countryCode"] |
[dwc:countryCode="CL", dwc:decimalLatitude="-41.0525925872862", dwc:decimalLongitude="-71.5310546742521": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="Coordinates are in Argentina, not Chile"] | |
[dwc:countryCode="ZX", dwc:decimalLatitude="-41.0525925872862", dwc:decimalLongitude="-71.5310546742521": Response.status=INTERNAL_PREREQUISTES_NOT_MET, Response.comment="Input field contains invalid values - ZX is not a valid ISO 3166-1-alpha-2 country code"] | |
Source | ALA, GBIF, iDigBio |
References |
|
Example Implementations (Mechanisms) | Kurator:geo_ref_qc |
Link to Specification Source Code | https://github.com/FilteredPush/geo_ref_qc/blob/master/src/main/java/org/filteredpush/qc/georeference/DwCGeoRefDQ.java#L80 |
Notes | dwc:coordinatePrecicision and dwc:coordinateUncertaintyInMeters (if present) imply a potential displacement of the provided coordinates. These two terms can be considered spatial buffers. Likewise, country polygons cannot be 100% accurate at all scales (Dooley 2005), so a spatial buffer of the country boundaries is justified. Taking the spatial buffers into account does however greatly complicate both the logic and the implementation of such tests. The same applies to potential conversion of the Spatial Reference System (SRS) of dwc:decimalLatitude and dwc:decimalLongitude to the SRS used in the bdq:sourceAuthority. |
Write welcome to Task Group 3 on the Read Me file of the Interest Group (https://tdwg.github.io/bdq/tg3/)
TestField | Value |
---|---|
GUID | 0aa03213-6205-48d1-85f7-23ec013ac272 |
Label | AMENDMENT_MINELEVATIONMAXELEVATION_TRANSPOSED |
Description | If "dwc:minimumElevationInMeters is greater than dwc:maximumElevationInMeters, can they be meaningfully swapped? |
TestType | Amendment |
Darwin Core Class | Location |
Information Elements ActedUpon | dwc:minimumElevationInMeters |
dwc:maximumElevationInMeters | |
Information Elements Consulted | |
Expected Response | TBA |
Data Quality Dimension | Conformance |
Term-Actions | MINELEVATIONMAXELEVATION_TRANSPOSED |
Parameter(s) | |
Source Authority | |
Specification Last Updated | 2024-02-22 |
Examples | [dwc:minimumElevationInMeters="256", dwc:maximumElevationInMeters="5": Response.status=SWAPPED_VALUES, Response.result=dwc:minimumElevationInMeters="5", dwc:maximumElevationInMeters="256", Response.comment="dwc:minimumElevationInMeters is greater than dwc:maximumElevationInMeters so they have been swapped"] |
[dwc:minimumElevationInMeters="25", dwc:maximumElevationInMeters="225": Response.status=NOT_CHANGED, Response.result="", Response.comment=""dwc:minimumElevationInMeters is lessr than dwc:maximumElevationInMeters so no changes made"] | |
Source | @Tasilee |
References | |
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes | Given that the reason for the transposition might be a transcription error in one or both elevation terms, erroneous transpositions whose validity could not be tested might be produced. For that reason it is not recommended to implement this amendment. |
TestField | Value |
---|---|
GUID | 0bb8297d-8f8a-42d2-80c1-558f29efe798 |
Label | VALIDATION_MINELEVATION_INRANGE |
Description | Is the value of dwc:minimumElevationInMeters within the Parameter range? |
TestType | Validation |
Darwin Core Class | Location |
Information Elements ActedUpon | dwc:minimumElevationInMeters |
Information Elements Consulted | |
Expected Response | INTERNAL_PREREQUISITES_NOT_MET if dwc:minimumElevationInMeters is EMPTY or the value is not a number; COMPLIANT if the value of dwc:minimumElevationInMeters is within the range of bdq:minimumValidElevationInMeters to bdq:maximumValidElevationInMeters inclusive; otherwise NOT_COMPLIANT |
Data Quality Dimension | Conformance |
Term-Actions | MINELEVATION_INRANGE |
Parameter(s) | bdq:minimumValidElevationInMeters |
bdq:maximumValidElevationInMeters | |
Source Authority | bdq:minimumValidElevationInMeters default = "-430" |
bdq:maximumValidElevationInMeters default = "8850" | |
Specification Last Updated | 2023-09-17 |
Examples | [dwc:minimumElevationInMeters="0": Response.status=RUN_HAS_RESULT, Response.result=COMPLIANT, Response.comment="dwc:minimumElevationInMeters is IN_RANGE"] |
[dwc:minimumElevationInMeters="-500": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="dwc:minimumElevationInMeters is NOT_IN_RANGE (<-430)"] | |
Source | ALA, GBIF |
References |
|
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes | We have rounded up the Parameter values. We are aware of sub-ice elevations in Antarctica to -3,500m and possible sampling in the atmosphere above the elevation of the top of Mt Everest that would fail this test but we support the odd false positive. |
TestField | Value |
---|---|
GUID | bd385eeb-44a2-464b-a503-7abe407ef904 |
Label | AMENDMENT_DCTYPE_STANDARDIZED |
Description | Propose amendment to the value of dc:type using the DCMI type vocabulary. |
TestType | Amendment |
Darwin Core Class | Record-level |
Information Elements ActedUpon | dc:type |
Information Elements Consulted | |
Expected Response | EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if the value of dc:type is EMPTY; AMENDED the value of dc:type if it can be unambiguously interpreted as a term name in the DCMI Type Vocabulary (see bdq:sourceAuthority); otherwise NOT_AMENDED |
Data Quality Dimension | Conformance |
Term-Actions | DCTYPE_STANDARDIZED |
Parameter(s) | |
Source Authority | bdq:sourceAuthority is "DCMI Type Vocabulary" {[http://purl.org/dc/terms/DCMIType]} {"DCMI Type Vocabulary List Of Terms" [https://www.dublincore.org/specifications/dublin-core/dcmi-type-vocabulary/2010-10-11/]} |
Specification Last Updated | 2024-03-20 |
Examples | [dc:type="event": Response.status=AMENDED, Response.result=dc:type="Event", Response.comment="dc:type contains an interpretable value"] |
[dc:type="X": Response.status=NOT_AMENDED, Response.result="", Response.comment="dc:type contains an uninterpretable value"] | |
Source | VertNet |
References |
|
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes | dc:type holds literals (e.g. PhysicalObject), while dcterms:type holds an IRI for the resource (e.g. http://purl.org/dc/dcmitype/PhysicalObject), see the Darwin Core RDF guide https://dwc.tdwg.org/rdf/#32-imported-dublin-core-terms-for-which-only-literal-objects-are-appropriate-normative. Implementations of this Amendment are expected be able to amend IRI values to the literals, as well as removing leading/trailing whitespace and correcting case errors in the literal. |
TestField | Value |
---|---|
GUID | 3f335517-f442-4b98-b149-1e87ff16de45 |
Label | VALIDATION_SCIENTIFICNAME_FOUND |
Description | Is there a match of the contents of dwc:scientificName with bdq:sourceAuthority? |
TestType | Validation |
Darwin Core Class | Taxon |
Information Elements ActedUpon | dwc:scientificName |
Information Elements Consulted | |
Expected Response | EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if dwc:scientificName is EMPTY; COMPLIANT if there is a match of the contents of dwc:scientificName with the bdq:sourceAuthority; otherwise NOT_COMPLIANT |
Data Quality Dimension | Conformance |
Term-Actions | SCIENTIFICNAME_FOUND |
Parameter(s) | bdq:sourceAuthority |
Source Authority | bdq:sourceAuthority default = "GBIF Backbone Taxonomy" {[https://doi.org/10.15468/39omei]} {API endpoint [https://api.gbif.org/v1/species?datasetKey=d7dddbf4-2cf0-4f39-9b2a-bb099caae36c&name=]} |
Specification Last Updated | 2023-09-17 |
Examples | [dwc:scientificName="Eucalyptus camaldulensis": Response.status=RUN_HAS_RESULT, Response.result=COMPLIANT, Response.comment="dwc:scientificName found in bdq:sourceAuthority"] |
[dwc:scientificName="Capulus intort": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="dwc:scientificName was not found in bdq:sourceAuthority"] | |
Source | ALA |
References |
|
Example Implementations (Mechanisms) | FP-Akka |
Link to Specification Source Code | |
Notes | The purpose of this test is to detect errors in the scientific name but is dependent on the abilities of the parsing of the bdq:sourceAuthority. For research users of biodiversity data doing quality assurance, VALIDATION_TAXON_UNAMBIGUOUS (4c09f127-737b-4686-82a0-7c8e30841590) handles their needs, but for curators of data sets doing quality control, this test provides a specific subset of targeted data cleaning, making it a valuable test to include for the quality control case. |
Just a reminder to everyone that we have two Seminar slots for Data Quality at TDWG2017 and that the Abstract deadline is 14 July (https://tdwg.github.io/conferences/2017/call-for-abstracts/)
We are also planning a workshop session during the week, and a closed meeting on the Sunday.
Just to inform everyone here that the Data Quality interest Group has established a mailing list on the TDWG listserve so that we can do a broad mail out without having to create a new issue on the GitHub. We would prefer to retain the Issues for actual GitHub-Like issues. You can join the listserve at (http://lists.tdwg.org/mailman/listinfo/tdwg-bdq)
Shortly (before end October), we will begin adding an issue for each of the TG2 Tests and Assertions, along with discussions already held in the spreadsheet. I urge you to participate in this exercise once they are posted so that we can finalise the tests and begin implementation.
TestField | Value |
---|---|
GUID | 6ce2b2b4-6afe-4d13-82a0-390d31ade01c |
Label | VALIDATION_COUNTRY_NOTEMPTY |
Description | Is there a value in dwc:country? |
TestType | Validation |
Darwin Core Class | Location |
Information Elements ActedUpon | dwc:country |
Information Elements Consulted | |
Expected Response | COMPLIANT if dwc:country is not EMPTY; otherwise NOT_COMPLIANT |
Data Quality Dimension | Completeness |
Term-Actions | COUNTRY_NOTEMPTY |
Parameter(s) | |
Source Authority | |
Specification Last Updated | 2023-09-17 |
Examples | [dwc:country="Eswatini": Response.status=RUN_HAS_RESULT, Response.result=COMPLIANT, Response.comment="dwc:country is not EMPTY"] |
[dwc:country="": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="dwc:country is EMPTY"] | |
Source | |
References |
|
Example Implementations (Mechanisms) | FilteredPush: geo_ref_qc |
Link to Specification Source Code | geo_ref_qc DwCGeoRefDQ.validationCountryNotEmpty |
Notes | Country is expected to be empty if material comes from the high seas, or from those portions of Antartica outside of any sovereign nation. |
TestField | Value |
---|---|
GUID | 620749b9-7d9c-4890-97d2-be3d1cde6da8 |
Label | AMENDMENT_COORDINATES_CONVERTED |
Description | Propose amendment to the value of dwc:geodeticDatum and potentially to dwc:decimalLatitude and/or dwc:decimalLongitude based on a conversion between coordinate reference systems. |
TestType | Amendment |
Darwin Core Class | Location |
Information Elements ActedUpon | dwc:decimalLatitude |
dwc:decimalLongitude | |
dwc:geodeticDatum | |
dwc:coordinateUncertaintyInMeters | |
dwc:coordinatePrecision | |
Information Elements Consulted | |
Expected Response | INTERNAL_PREREQUISITES_NOT_MET if dwc:decimalLatitude is EMPTY or does not have a valid value, or dwc:decimalLongitude is EMPTY or does not have a valid value, or dwc:geodeticDatum is EMPTY or does not contain an interpretable value; AMENDED if the values of dwc:decimalLatitude, dwc:decimalLongitude and dwc:geodeticDatum are changed based on a conversion between the coordinate reference systems as specified by dwc:geodeticDatum and bdq:targetCRS, and, if dwc:coordinateUncertaintyInMeters was an interpretable value, the uncertainty from the conversion is added to it, and the value of dwc:coordinatePrecision is provided from the conversion result; otherwise NOT_AMENDED. |
Data Quality Dimension | Conformance |
Term-Actions | COORDINATES_CONVERTED |
Parameter(s) | bdq:targetCRS |
Source Authority | bdq:targetCRS default = "EPSG:4326" {[https://epsg.org]} {EPSG Endpoint for translations [https://epsg.io/transform]} |
Specification Last Updated | 2023-09-18 |
Examples | [dwc:decimalLatitude="-23.712", dwc:decimalLongitude="139.923", dwc:geodeticDatum="AGD66", dwc:coordinateUncertaintyInMeters="", dwc:coordinatePrecision="": Response.status=AMENDED, Response.result=dwc:decimalLatitude="-23.7105001", dwc:decimalLongitude="139.924185", dwc:geodeticDatum="EPSG:4326", dwc:coordinateUncertaintyInMeters="", dwc:coordinatePrecision="6", Response.comment="Input fields contain interpretable values: xform using "https://epsg.io/transform#s_srs=4202&t_srs=4326&x=139.9230000&y=-23.7120000" "] |
[dwc:decimalLatitude="-93.712", dwc:decimalLongitude="139.923", dwc:geodeticDatum="GDA94", dwc:coordinateUncertaintyInMeters="", dwc:coordinatePrecision="": Response.status=NOT_AMENDED, Response.result=, Response.comment="dwc:decimalLatitude was out of range"] | |
Source | ALA, GBIF |
References |
|
Example Implementations (Mechanisms) | |
Link to Specification Source Code | |
Notes | This test relates only to EPSG codes applying to coordinate reference systems where the coordinate system is EPSG:6422 (Ellipsoidal 2D CS. Axes: latitude, longitude. Orientations: north, east. UoM: degree), or EPSG:6423 (Ellipsoidal 3D CS. Axes: latitude, longitude, ellipsoidal height. Orientations: north, east, up. UoM: degree, degree, metre.). Any amendment has implications for dwc:coordinateUncertaintyInMeters and dwc:coordinatePrecision. If the dwc:coordinateUncertaintyInMeters is EMPTY or is not interpretable, this amendment should not provide a dwc:coordinateUncertaintyInMeters. If the dwc:coordinateUncertaintyInMeters is not EMPTY and is valid, this amendment should add the uncertainty contributed by the conversion to the value of dwc:coordinateUncertaintyInMeters. The amended dwc:coordinatePrecision should be the precision of coordinates as provided after the conversion, ideally this should be 0.0000001, reflecting the seven digits of precision required to reverse a coordinate transformation without loss of information at the scale of one meter. If dwc:geodeticDatum specifies the same CRS for dwc:decimalLatitude and dwc:decimalLongitude as bdq:targetCRS (e.g., if dwc:geodeticDatum has either the value "WGS84" or "EPSG:4326" and the bdq:targetCRS is "EPSG:4326"), then the coordinates are assumed to be in the target CRS and the Response.status is NOT_AMENDED. |
The "how you can contribute" section of the README currently only has boilerplate text:
Depending on where group interaction takes place (GitHub, mailing list), provide some guidance on how interested parties can follow group activities (e.g. watch this repository, join mailing list) and how they can contribute.
Please update this to how people can actually contribute.
The DQIG Annual Report for 2017 has been published on the GitHub Wiki and is available for perusal.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.