Code Monkey home page Code Monkey logo

Comments (56)

chicoreus avatar chicoreus commented on August 24, 2024 2

Also worth referencing EDTF https://www.loc.gov/standards/datetime/ which is incorporated into ISO:8601-2 (2019), and looking for somewhere with a good summary of 8601/8601-2 to point people at.

from bdq.

Tasilee avatar Tasilee commented on August 24, 2024 2

I would prefer

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1; FILLED_IN EMPTY terms dwc:year, dwc:month, dwc:day, dwc:startDayOfYear, dwc:endDayOfYear if any were unambiguously interpreted from values in dwc:eventDate, otherwise NOT_AMENDED

but then the implementation would then be dependent on the Notes, which to me is a no no. What about something like-

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1; FILLED_IN dwc:day if it is EMPTY and dwc:eventDate can be resolved to a single day, FILLED_IN dwc:month if it is EMPTY and dwc:eventDate can be resolved to a single month, FILLED_IN dwc:year if it is EMPTY and dwc:eventDate can be resolved to a single year, FILLED_IN dwc:startDayOfYear and dwc:endDayOfYear if they are EMPTY and they can be unambiguously interpreted from dwc:eventDate; otherwise NOT_AMENDED.

?

The spanning years Note is then OK.

from bdq.

chicoreus avatar chicoreus commented on August 24, 2024 2

Added missing "or finer" from clause 2 for month. Now consistent with day and year clauses, otherwise implemetors would not fill in month if the event date had a precision of one day.

from bdq.

chicoreus avatar chicoreus commented on August 24, 2024 1

I propose changing the description to: One or more empty component terms of the dwc:Event class (dwc:year, dwc:month, dwc:day, dwc:startDayOfYear, dwc:endDayOfYear) have been filled in from a valid value in the term dwc:eventDate.

The concept reflected in in the note " and only if dwc:eventDate has a valid ISO 8601:2004(E) date" should be reflected in the description. It is an important point for implementers.

from bdq.

chicoreus avatar chicoreus commented on August 24, 2024 1

There is an inter-amendment workflow dependency to note here - this amendment should run after all other amendments that may affect the value of eventDate. (i.e. fill in the event date from the verbatimEventDate, then fill in year/month/day/startDayOfYear/endDayOfYear from the interpreted eventDate value, etc.). I've added a note to this effect to the prerequisites.

from bdq.

chicoreus avatar chicoreus commented on August 24, 2024 1

We should note that time is not included here as it is not considered core. In working with an implementation of this, I've found extracting time from eventDate to be fraught with all sorts of concerns (which aren't going to be core concerns), including handling time zone and handling times on eventDates which involve ranges. Non-trivial to specify appropriate behaviors, and those aren't core. We are much safer not having this particular test propose to fill in eventTime.

from bdq.

chicoreus avatar chicoreus commented on August 24, 2024 1

We do need to specify if endDayOfYear is expected to be filled in if the eventDate represents a single day.

from bdq.

Tasilee avatar Tasilee commented on August 24, 2024 1

Given comments from @tucotuco, then maybe

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains a value that is not valid according to ISO 8601-1:2019; FILLED_IN one or more EMPTY terms dwc:year, dwc:month, dwc:day, dwc:startDayOfYear, dwc:endDayOfYear if they can be unambiguously interpreted from values in dwc:eventDate, and dwc:eventDate is wholly within one year; otherwise NOT_AMENDED

I have changed it to this for now and will align the other occurrences if there is agreement.

from bdq.

Tasilee avatar Tasilee commented on August 24, 2024 1

I've edited the Expected Response according to @tucotuco suggestion:

From

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1:2019; FILLED_IN one or more EMPTY terms dwc:year, dwc:month, dwc:day, dwc:startDayOfYear, dwc:endDayOfYear if they can be unambiguously interpreted from values in dwc:eventDate, and dwc:eventDate is wholly within one year; otherwise NOT_AMENDED

to

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1 or dwc:eventDate is not wholly within one year; FILLED_IN one or more EMPTY terms dwc:year, dwc:month, dwc:day, dwc:startDayOfYear, dwc:endDayOfYear if any were unambiguously interpreted from values in dwc:eventDate; otherwise NOT_AMENDED

and updated the References

from bdq.

Tasilee avatar Tasilee commented on August 24, 2024 1

I have updated the ISO reference link.

from bdq.

chicoreus avatar chicoreus commented on August 24, 2024 1

We may be able to handle consistency with notes, but the bit ", and dwc:eventDate is wholly within one year" may not hold.

Here is a way we could be explicit in the specification about expectations, changing from:

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1 or dwc:eventDate is not wholly within one year; FILLED_IN one or more EMPTY terms dwc:year, dwc:month, dwc:day, dwc:startDayOfYear, dwc:endDayOfYear if any were unambiguously interpreted from values in dwc:eventDate, and dwc:eventDate is wholly within one year; otherwise NOT_AMENDED

To:

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1 or dwc:eventDate is not wholly within one year; FILLED_IN one or more EMPTY terms dwc:year, dwc:month, dwc:day, dwc:startDayOfYear, dwc:endDayOfYear if any were unambiguously interpreted from values in dwc:eventDate, under the following rules: if dwc:eventDate is a range of more than one day or lacks precision to day, dwc:day is not populated; if dwc:eventDate is a range spanning than one month, or lacks precision to month, dwc;month is not populated; if dwc:eventDate is a range spanning more than one year dwc:year is not populated; dwc;startDayOfYear and endDayOfYear are to be filled in if they can be unambiguously interpreted from dwc:eventDate; otherwise NOT_AMENDED

Noting that for ranges that span a year boundary, startDayOfYear may be larger than endDayOfYear, and that startDayOfYear and endDayOfYear should be populated if eventDate has either reduced precision (e.g. 1872), or is a range (1872-04-06/1872-08-12).

ISO 8601-1 distinguishes between reduced precision dates and date ranges (thus 1984 and 1984-01-01/1984-12-31 are not the same thing), but biodiversity data is stored in a wide variety of data structures (year, month, day integer fields, pairs of date fields for start and end, structured strings that can take EDTF like strings with explicit markers of uncertainty (1872--)), and go through various transformations with various software including libraries that don't distinguish between reduced precision dates and ranges), and we have low confidence that the meaning of dwc:eventDate=1984 is different from the meaning of dwc:eventDate=1984-01-01/1984-12-31.

But, we might want to put only the following in the specification;

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1 or dwc:eventDate is not wholly within one year; FILLED_IN one or more EMPTY terms dwc:year, dwc:month, dwc:day, dwc:startDayOfYear, dwc:endDayOfYear if any were unambiguously interpreted from values in dwc:eventDate, otherwise NOT_AMENDED

And then put into notes:
Fill in values of empty terms under the following rules: if dwc:eventDate is a range of more than one day or lacks precision to day, dwc:day is not populated; if dwc:eventDate is a range spanning than one month, or lacks precision to month, dwc;month is not populated; if dwc:eventDate is a range spanning more than one year dwc:year is not populated; dwc;startDayOfYear and endDayOfYear are to be filled in if they can be unambiguously interpreted from dwc:eventDate;

from bdq.

chicoreus avatar chicoreus commented on August 24, 2024 1

Working from @Tasilee's comment and the notes in #67 (comment) how about (though we could restructure following the precision of eventDate as in #67 (comment) instead):

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1; FILLED_IN if any of the following EMPTY values are populated: 1) day from dwc:eventDate is proposed as the value for dwc:day if dwc:day is EMPTY and dwc:eventDate has a precision of a day or finer, 2) month from dwc:eventDate is proposed as the value for dwc:month if dwc:month is EMPTY and dwc:eventDate has a precision of a single month, 3) year from dwc:eventDate is proposed as the value for dwc:year if dwc:year is EMPTY and dwc:eventDate has a precision of a single year or finer and is within a single year, 4) values for dwc:startDayOfYear and dwc:endDayOfYear are proposed if they are EMPTY and dwc:eventDate has a precision of a day or better; otherwise NOT_AMENDED.

I'm not sure if the form "precision of a day or finer" should be that or "precision of a day or finer and spans a single day or less" for day and month, more confident that the later should be the case for year. @tucotuco?

from bdq.

Tasilee avatar Tasilee commented on August 24, 2024 1

My worry here are references to "day", "month" and "year". A first? My inclination would be to the form-

FILLED_IN 1) dwc:day from dwc:eventDate if dwc:day is EMPTY and dwc:eventDate has a precision of a day or finer...overall

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1; FILLED_IN 1) dwc:day from day from dwc:eventDate if dwc:day is EMPTY and dwc:eventDate has a precision of a day or finer, 2) dwc:month from dwc:eventDate if dwc:month is EMPTY and dwc:eventDate has a precision of a single month, 3) dwc:year from dwc:eventDate if dwc:year is EMPTY and dwc:eventDate has a precision of a single year or finer and is within a single year, 4) dwc:startDayOfYear and dwc:endDayOfYear if they are EMPTY and dwc:eventDate has a precision of a day or better; otherwise NOT_AMENDED.

from bdq.

ArthurChapman avatar ArthurChapman commented on August 24, 2024 1

I am happy with that @Tasilee but note minor correction (1) dwc.day from dwc:eventDate.... (was 1) dwc:day from day from dwc:eventDate...

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1; FILLED_IN 1) dwc:day from dwc:eventDate if dwc:day is EMPTY and dwc:eventDate has a precision of a day or finer, 2) dwc:month from dwc:eventDate if dwc:month is EMPTY and dwc:eventDate has a precision of a single month, 3) dwc:year from dwc:eventDate if dwc:year is EMPTY and dwc:eventDate has a precision of a single year or finer and is within a single year, 4) dwc:startDayOfYear and dwc:endDayOfYear if they are EMPTY and dwc:eventDate has a precision of a day or better; otherwise NOT_AMENDED.

from bdq.

chicoreus avatar chicoreus commented on August 24, 2024 1

How about:

FILLED_IN if any of (1)...

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1; FILLED_IN if any of (1) dwc:day from dwc:eventDate if dwc:day is EMPTY and dwc:eventDate has a precision of a day or finer and is within a single day, (2) dwc:month from dwc:eventDate if dwc:month is EMPTY and dwc:eventDate has a precision of a single month or finer and is within a single month, (3) dwc:year from dwc:eventDate if dwc:year is EMPTY and dwc:eventDate has a precision of a single year or finer and is within a single year, (4) dwc:startDayOfYear and dwc:endDayOfYear if they are EMPTY and dwc:eventDate has a precision of a day or better; otherwise NOT_AMENDED.

from bdq.

iDigBioBot avatar iDigBioBot commented on August 24, 2024

Comment by Paul Morris (@chicoreus) migrated from spreadsheet:
One way of simplifying the core test suite would be to identify a small set of primary fields in cases where Darwin Core allows for multiple representations of the data (Event fields are the clearest example of this), and only propose amendments that work to fill in the primary fields from secondary fields (eventDate from day, month, year, verbatimEventDate, startDayOfYear, endDayOfYear, eventTime), and not include in the core suite ammendments that fill in secondary fields from the primary fields.

from bdq.

iDigBioBot avatar iDigBioBot commented on August 24, 2024

Comment by Arthur Chapman (@ArthurChapman) migrated from spreadsheet:
I agree with Paul ghere

from bdq.

iDigBioBot avatar iDigBioBot commented on August 24, 2024

Comment by Paula Zermoglio (@pzermoglio) migrated from spreadsheet:
Agree too. However, you may piss some people off, eg those that prefer using YMD instead of eventDate?

from bdq.

ArthurChapman avatar ArthurChapman commented on August 24, 2024

@JohnW wanted to be able to back populate - at least for year - as many people want to extract just the year from the data.

from bdq.

chicoreus avatar chicoreus commented on August 24, 2024

The danger with backpopulating is that eventDate is capable of handling richer information than the atomic fields (date ranges and date ranges which span more than one year). A consumer who wants to simply obtain the year from dwc:year does so at their own peril if dwc:eventDate contains a date range which spans more than one year. That, however, is a let the consumer beware kind of issue. We shouldn't advocate back populating because people may want to use the data, as it potentially is unfit for their use, but I'm entirely in favor of back populating in order to make data sets consistent in their presentation - filling in all fields that can be filled in.

from bdq.

ArthurChapman avatar ArthurChapman commented on August 24, 2024

@chicoreus - wrt to workflow - I have added "After #33, #49, #86, #93, #132" for test #52 - see the circulated workflow document

from bdq.

ArthurChapman avatar ArthurChapman commented on August 24, 2024

@chicoreus - fully agree wrt note on time.

from bdq.

Tasilee avatar Tasilee commented on August 24, 2024

I have reviewed the parameters and notes (all good) and believe we have a useable outcome.

from bdq.

chicoreus avatar chicoreus commented on August 24, 2024

See comment in #131 about whether this test should specify not filling in endDayOfYear in cases where eventDate is a range of years.

from bdq.

ArthurChapman avatar ArthurChapman commented on August 24, 2024

@chicoreus. In discussion in Gainesville, I think I said I saw less value in filling all these other fields from eventDate than going the other way and making sure eventDate was filled in wherever possible. So, filling in endDayOfYear etc. I see being of lower value, and I would be happy with your suggestion. However, those that advocated for this test should be better commenting on it than I.

from bdq.

Tasilee avatar Tasilee commented on August 24, 2024

Reading your comment on #131, I agree with your logic @chicoreus. How would you work the Expected Response?

INTERNAL_PREREQUESITES_NOT_MET if the field dwc:eventDate is EMPTY or does not contain a valid ISO 8601-1:2019 date; AMENDED if one or more EMPTY terms of the dwc:Event class (dwc:year, dwc:month, dwc:day, dwc:startDayOfYear, dwc:endDayOfYear) have been filled in from a valid unambiguously interpretable value in dwc:eventDate, unless dwc:eventDate spans years in which case dwc:endDayOfyear is not FILLED_IN; otherwise NOT_CHANGED?

from bdq.

ArthurChapman avatar ArthurChapman commented on August 24, 2024

@Tasilee. We don't have a term for "NOT_FILLED_IN so I would say

INTERNAL_PREREQUESITES_NOT_MET if the field dwc:eventDate is EMPTY or does not contain a valid ISO 8601-1:2019 date; AMENDED if one or more EMPTY terms of the dwc:Event class (dwc:year, dwc:month, dwc:day, dwc:startDayOfYear, dwc:endDayOfYear) have been filled in from a valid unambiguously interpretable value in dwc:eventDate and eventDate is wholly within the one year; otherwise, or if dwc:eventDate spans more than one year, NOT_CHANGED?

from bdq.

Tasilee avatar Tasilee commented on August 24, 2024

@ArthurChapman: Much better. Editing.

from bdq.

ArthurChapman avatar ArthurChapman commented on August 24, 2024

Or, perhaps even simpler - putting in into positive and being consistent

INTERNAL_PREREQUESITES_NOT_MET if the field dwc:eventDate is EMPTY or does not contain a valid ISO 8601-1:2019 date; AMENDED if one or more EMPTY terms of the dwc:Event class (dwc:year, dwc:month, dwc:day, dwc:startDayOfYear, dwc:endDayOfYear) have been filled in from a valid unambiguously interpretable value in dwc:eventDate and eventDate is wholly within the one year; otherwise NOT_CHANGED

from bdq.

Tasilee avatar Tasilee commented on August 24, 2024

@ArthurChapman: Yep

from bdq.

timrobertson100 avatar timrobertson100 commented on August 24, 2024

Dealing with timezones is a real issue faced when integrating and accessing data. Yes, it is difficult but clarifying expected behavior when a) data spans timezones, b) dealing with datasets holding date times with and without timezone and c) when the timezone of the consumer is not known would be helpful. It might result in recommendations to store a localDateTime version as well as a UTC-normalized version.

This should also take into consideration the expectations of simple human observations (e.g. a naturalist record from 07:30 local time on 1st January 2019 in New Zealand being returned in a 2019 search and not a 2018 search) as well as those of machine recorders where high-frequency samples are taken and stored in UTC regardless of location even though the moving organism is crossing timezones.

from bdq.

chicoreus avatar chicoreus commented on August 24, 2024

@timrobertson100 discussion in TG2 call today: (1) time was decided as out of scope for the TG2 tests early on, and we'd have to add in all the complexities of time to address the timezone concern. (2) The tests, in particular this one, are asking about the representation of date in a single record, and don't involve comparisons between dates in different records. If dwc:eventTime is included in the New Zealand record you discuss, a consumer of the data is able to interpret which year to place that record into for purposes of search, we are seeing this as an independent question from the internal consistency of the terms in the record itself.

from bdq.

chicoreus avatar chicoreus commented on August 24, 2024

Discussion in call and @tucotuco 's observation that Darwin Core is vauge on whether dwc:endDayOfYear is tied to the end of a date range, and the thought that we could put a stake in the ground towards expectations for this case of endDayOfYear meaning the day of the year of the end of a range expressed in eventDate. Thus, @Tasilee here's a proposal for changing the specification of this test:

INTERNAL_PREREQUESITES_NOT_MET if the field dwc:eventDate is EMPTY or does not contain a valid ISO 8601-1:2019 date; AMENDED if one or more EMPTY terms of the dwc:Event class (dwc:year, dwc:month, dwc:day, dwc:startDayOfYear, dwc:endDayOfYear) have been filled in from a valid unambiguously interpretable value in dwc:eventDate; otherwise NOT_CHANGED

And Note:
Only fields that are empty will be amended, and only if dwc:eventDate has a valid ISO 8601-1:2019 date. The dwc:eventDate is the canonical form of the event date (it is the first trusted form). If event date does not contain a range, dwc:startDayOfYear = dwc:endDayOfYear. Time (as compared to date) is not deemed a CORE component. NB Run this amendment after any other amendment which may affect dwc:eventDate. If eventDate contains a date range, dwc:startDayOfYear is to be interpreted as the day of year of the start of the date range, and dwc:endDayOfYear is to be interpreted as the day of the year of the end of the date range, thus endDayOfYear could be smaller than startDayOfYear as in 2015-12-15/2016-01-15.

from bdq.

Tasilee avatar Tasilee commented on August 24, 2024

Thanks @chicoreus. It is simpler but certainly needs the Note for clarification. Is everyone else happy with this before I amend?

In regards ISO 8601, we can reference it in the specs as discussed but I am tempted to add https://en.wikipedia.org/wiki/ISO_8601 to the references everywhere (in each test) that requires it. What do you think? Reasoning: We all agreed on the benefits of having our table as self-contained as possible (accepting a move from non-canonical to Notes) and access to details of ISO standards is not a simple end-point or free.

from bdq.

Tasilee avatar Tasilee commented on August 24, 2024

Agreed. I have added these refs here, but assume they need to go everywhere we have a TIME tab?

from bdq.

Tasilee avatar Tasilee commented on August 24, 2024

Changed "AMENDED" to "FILLED_IN" in accordance with discussions April 16.

from bdq.

Tasilee avatar Tasilee commented on August 24, 2024

Given recent discussions, I have changed

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to bdq:sourceAuthority; FILLED_IN one or more EMPTY terms dwc:year, dwc:month, dwc:day, dwc:startDayOfYear, dwc:endDayOfYear if they can be unambiguously interpreted from values in dwc:eventDate, and dwc:eventDate is wholly within one year; otherwise NOT_AMENDED

to

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid ISO 8601-1 date; FILLED_IN one or more EMPTY terms dwc:year, dwc:month, dwc:day, dwc:startDayOfYear, dwc:endDayOfYear if they can be unambiguously interpreted from values in dwc:eventDate, and dwc:eventDate is wholly within one year; otherwise NOT_AMENDED

...and I have removed the ref to bdq:sourceAuthority.

from bdq.

tucotuco avatar tucotuco commented on August 24, 2024

This may be too picky, but I would reword to:

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains a value that is not valid according to ISO 8601-1; FILLED_IN one or more EMPTY terms dwc:year, dwc:month, dwc:day, dwc:startDayOfYear, dwc:endDayOfYear if they can be unambiguously interpreted from values in dwc:eventDate, and dwc:eventDate is wholly within one year; otherwise NOT_AMENDED

from bdq.

ArthurChapman avatar ArthurChapman commented on August 24, 2024

I like that wording @tucotuco

from bdq.

Tasilee avatar Tasilee commented on August 24, 2024

To standardize the terminology, I am editing occurrences of "not valid" in the Expected Responses to "invalid"

from bdq.

ArthurChapman avatar ArthurChapman commented on August 24, 2024

That conforms to the Vocabulary #152 where we have "Invalid" ["Where the Data Quality Dimension: Conformance (q.v.) is not satisfied due to Information Elements (q.v.) containing non-standard values, values outside of an acceptable range, or values unable to be found in a bdq:sourceAuthority (q.v.)."] but not "not valid" (or NOT_VALID)

from bdq.

Tasilee avatar Tasilee commented on August 24, 2024

Changed 'NOT AMENDED' example.

Also awaiting vote or comment on #52 (comment) from @ArthurChapman and @tucotuco

from bdq.

ArthurChapman avatar ArthurChapman commented on August 24, 2024

Can't just vote as there are two options! Also both options have some errors need - "...range spanning more than one month..."

Both options have a contradiction - if the year spans > one year, it can't be both INTERNAL_PREREQUISITES_NOTMET and FILLED_IN.

I would vote for a modification of the second option, i.e.

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1; FILLED_IN one or more EMPTY terms dwc:year, dwc:month, dwc:day, dwc:startDayOfYear, dwc:endDayOfYear if any were unambiguously interpreted from values in dwc:eventDate, otherwise NOT_AMENDED

And then put into notes:
Fill in values of empty terms under the following rules: if dwc:eventDate is a range of more than one day or lacks precision to day, dwc:day is not populated; if dwc:eventDate is a range spanning more than one month, or lacks precision to month, dwc;month is not populated; if dwc:eventDate is a range spanning more than one year dwc:year is not populated; dwc;startDayOfYear and endDayOfYear are to be filled in if they can be unambiguously interpreted from dwc:eventDate noting that for ranges that span a year boundary, startDayOfYear may be larger than endDayOfYear.

from bdq.

chicoreus avatar chicoreus commented on August 24, 2024

See also the discussion in #67 and notes from TG2 call by @ArthurChapman therin #67 (comment)

from bdq.

ArthurChapman avatar ArthurChapman commented on August 24, 2024

Looks better - getting there

from bdq.

chicoreus avatar chicoreus commented on August 24, 2024

@Tasilee I like that phrasing.

from bdq.

Tasilee avatar Tasilee commented on August 24, 2024

I have edited the Expected response (and Specification Last Updated) accordingly. NEEDS WORK??

from bdq.

ArthurChapman avatar ArthurChapman commented on August 24, 2024

I think NEEDS WORK can be turned off.

Further to a post elsewhere; we have in different tests:

  • (1), (2), (3)
  • 1), 2), 3)
  • a), b), c)
  • others?

from bdq.

Tasilee avatar Tasilee commented on August 24, 2024

See #43 (comment)

from bdq.

chicoreus avatar chicoreus commented on August 24, 2024

Updated phrasing of the note, Amendments do not amend data, they propose changes (in Response.result, with a Response.status of FILLED_IN (only FILLED_IN in this case) or AMENDED). We need to be careful that we don't use these constants as verbs.

from bdq.

chicoreus avatar chicoreus commented on August 24, 2024

See discussion in #67 and issue #204, likely needs a little bit of work to be consistent with the related tests.

from bdq.

chicoreus avatar chicoreus commented on August 24, 2024

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1; FILLED_IN (1) dwc:day from dwc:eventDate if dwc:day is EMPTY and dwc:eventDate has a precision of a day or finer and is within a single day, (2) dwc:month from dwc:eventDate if dwc:month is EMPTY and dwc:eventDate has a precision of a single month or finer and is within a single month, (3) dwc:year from dwc:eventDate if dwc:year is EMPTY and dwc:eventDate has a precision of a single year or finer and is within a single year, (4) dwc:startDayOfYear and dwc:endDayOfYear if they are EMPTY and dwc:eventDate has a precision of a day or better; otherwise NOT_AMENDED.

from bdq.

Tasilee avatar Tasilee commented on August 24, 2024

Splitting bdqffdq:Information Elements into "Information Elements ActedUpon" and "Information Elements Consulted". Also changed "Field" to "TestField" and "Output Type" to "TestType".

from bdq.

chicoreus avatar chicoreus commented on August 24, 2024

Current evaluation of validation data dataid 320 (from @Tasilee spreadsheet v68) suggests the numbered clauses in FILLED_IN need to be separated by "or":

320 #52 Fail got FILLED_IN expected NOT_AMENDED Provided value for dwc:eventDate [2007-03-01T13:00:00Z/2008-05-11T15:30:00Z] represents a range of more than one year.|Provided dwc:eventDate [2007-03-01T13:00:00Z/2008-05-11T15:30:00Z] spans more than one year, can't fill in dwc:year.|Added startDayOfYear [60] from start day of eventDate [2007-03-01T13:00:00Z/2008-05-11T15:30:00Z].|Added endDayOfYear [132] from end day of eventDate [2007-03-01T13:00:00Z/2008-05-11T15:30:00Z].

Interpretation in implementation (which is in line with the expectations of FILLED_IN) is that if any of the elements are FILLED_IN then that is used as the Response.status, only if no elements are filled in, then NOT_AMENDED is used.

from bdq.

Tasilee avatar Tasilee commented on August 24, 2024

That seems reasonable. I've edited DataID 320 to FILLED_IN

Also changed Expected Response from

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1; FILLED_IN (1) dwc:day from dwc:eventDate if dwc:day is EMPTY and dwc:eventDate has a precision of a day or finer and is within a single day, (2) dwc:month from dwc:eventDate if dwc:month is EMPTY and dwc:eventDate has a precision of a single month or finer and is within a single month, (3) dwc:year from dwc:eventDate if dwc:year is EMPTY and dwc:eventDate has a precision of a single year or finer and is within a single year, (4) dwc:startDayOfYear and dwc:endDayOfYear if they are EMPTY and dwc:eventDate has a precision of a day or better; otherwise NOT_AMENDED.

to

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1; (FILLED_IN dwc:day from dwc:eventDate if dwc:day is EMPTY and dwc:eventDate has a precision of a day or finer and is within a single day. FILLED_IN dwc:month from dwc:eventDate if dwc:month is EMPTY and dwc:eventDate has a precision of a single month or finer and is within a single month. FILLED_IN dwc:year from dwc:eventDate if dwc:year is EMPTY and dwc:eventDate has a precision of a single year or finer and is within a single year. FILLED_IN dwc:startDayOfYear and dwc:endDayOfYear if they are EMPTY and dwc:eventDate has a precision of a day or better) otherwise NOT_AMENDED.

We can't use 'OR' because that doesn't allow for multiple fills. Other option is to qualify 'NOT_AMENDED' to something like

If all of dwc:day, dwc:month, dwc:year, dwc:startDayOfYer and dwc:endDayOfYear were not FILLED_IN, then NOT_AMENDED.

from bdq.

Tasilee avatar Tasilee commented on August 24, 2024

Looks good to me @chicoreus

Changed

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1; FILLED_IN (1) dwc:day from dwc:eventDate if dwc:day is EMPTY and dwc:eventDate has a precision of a day or finer and is within a single day, (2) dwc:month from dwc:eventDate if dwc:month is EMPTY and dwc:eventDate has a precision of a single month or finer and is within a single month, (3) dwc:year from dwc:eventDate if dwc:year is EMPTY and dwc:eventDate has a precision of a single year or finer and is within a single year, (4) dwc:startDayOfYear and dwc:endDayOfYear if they are EMPTY and dwc:eventDate has a precision of a day or better; otherwise NOT_AMENDED.

to

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1; FILLED_IN if any of (1) dwc:day from dwc:eventDate if dwc:day is EMPTY and dwc:eventDate has a precision of a day or finer and is within a single day, (2) dwc:month from dwc:eventDate if dwc:month is EMPTY and dwc:eventDate has a precision of a single month or finer and is within a single month, (3) dwc:year from dwc:eventDate if dwc:year is EMPTY and dwc:eventDate has a precision of a single year or finer and is within a single year, (4) dwc:startDayOfYear and dwc:endDayOfYear if they are EMPTY and dwc:eventDate has a precision of a day or better; otherwise NOT_AMENDED.

and updated 'Specification Last Updated'

from bdq.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.