metanorma / pubid-nist Goto Github PK

License: BSD 2-Clause "Simplified" License

Ruby 100.00%

pubid-nist's Introduction

NIST publication identifiers ("NIST PubID")

Purpose

NIST publications are numbered according to an identification scheme.

This gem implements a mechanism to parse and utilize NIST publication identifiers provided in the NIST-Tech-Pubs repository.

This gem is introduced in the Metanorma blog post "Introducing NIST PubID, and the migration of publication identifiers" on 2022-01-09.

For further details on the Metanorma work on the NIST PubID, refer to the "Ribose comments to Publication Identifier Syntax for NIST Technical Series Publications" document.

Historic identifier patterns

There are at least three major "pattern series" of identifiers due to historical reasons:

NIST publications produced by the PubID scheme (202X-)
NIST publications produced prior to the PubID scheme (1988-)
NBS publications, produced under the National Bureau of Standards (the previous name of NIST, 1901 to 1988)

This implementation attempts to support all three types of publication identifier patterns.

NIST PubID

NIST is currently developing a unified publication identifier scheme ("PubID") for their technical series of publications, which allows machine-readable identifier parts.

This scheme is described in “Publication Identifier Syntax for NIST Technical Series Publications” authored by Jim Foti of the CSD, ITL, and Kathryn Miller and Kate Bucher of the ISO, Management Resources.

Note	This scheme is still at proposal stage.

This gem implements the NIST PubID scheme and allows the conversion to and from historic patterns.

Use cases to support

PubID:

Given a PubID (full-form title, abbreviated-form title, short-form title), generate the corresponding machine-readable (MR) PubID format
Given a MR PubID:
1. generate the full-form title
2. generate the abbreviated-form title
3. generate short-form title
4. generate the DOI (which is just DOI ID + MR)

Historic compatibility:

Parse a historic publication identifier into its components:
1. output the original publication identifier (i.e. store the contents in the object)
2. output a MR PubID
Parse a historic DOI into a PubID:
1. output the original DOI (i.e. store the contents in the object)
2. output a MR PubID

CLI conversion tool

We implemented an open-source conversion tool that extracts the required PubID data elements from existing NIST Tech Pubs metadata, such as the legacy identifier, DOI, edition and publication date information, to generate the new PubID. pubid-nist provides a CLI (Command-Line Interface) and a Ruby library that can be used to create and manipulate PubID objects.

Installation

The only prerequisite is to have Ruby installed. Please refer to the official Ruby installation guide.

The pubid-nist tool can be installed as follows.

$ gem install pubid-nist

Now you should be able to use the pubid-nist command.

Usage

Installation

The only prerequisite is to have Ruby installed. Please refer to the official Ruby installation guide.

The pubid-nist tool can be installed as follows.

$ gem install pubid-nist

Now you should be able to use the pubid-nist command.

When called without arguments (or as pubid-nist help) the help screen will be shown.

$ pubid-nist
Commands:
  pubid-nist convert         # Convert legacy NIST Tech Pubs ID to NIST PubID
  pubid-nist help [COMMAND]  # Describe available commands or one specific command
  pubid-nist report          # Create report for NIST Tech Pubs database (fetches from GitHub)

Converting a legacy identifier to NIST PubID

The command pubid-nist provides a convert subcommand that converts a legacy Nist Tech Pubs identifier into the NIST PubID format.

Here’s how it can be used:

$ pubid-nist help convert
Usage:
  pubid-nist convert

Options:
  -s, [--style=STYLE]    # Convert to PubID style (short|long|mr|abbrev)
                         # Default: short
  -f, [--format=FORMAT]  # Render in format (JSON, string)
                         # Default: string

Convert legacy NIST Tech Pubs ID to NIST PubID

$ pubid-nist convert "NIST SP 800-53a"
NIST SP 800-53A
$ pubid-nist convert "NIST SP 800-57p1r3"
NIST SP 800-57pt1r3

The convert command also supports DOI conversion.

$ pubid-nist convert "NIST.SP.800-57p1r3"
NIST SP 800-57pt1r3

In addition to outputting PubID short style, we can also output other styles and formats of the resulting PubID.

$ pubid-nist convert -s mr "NIST SP 800-53a"
NIST.SP.800-53A
$ pubid-nist convert -s long -f json "NIST SP 800-53a" | jq
{
  "styles": {
    "short": "NIST SP 800-53A",
    "abbrev": "Natl. Inst. Stand. Technol. Spec. Publ. 800-53A",
    "long": "National Institute of Standards and Technology Special Publication 800-53A",
    "mr": "NIST.SP.800-53A"
  },
  "publisher": "NIST",
  "serie": "NIST SP",
  "code": "800-53A"
}

Generating the bulk NIST Tech Pubs migration report

This is the nice part — a single command that generates the full table of converted PubIDs from the NIST Tech Pubs database, comprising of 19,333 entries.

The report command can be used as follows:

$ pubid-nist help report
Usage:
  pubid-nist report

Options:
  [--csv], [--no-csv]  # Export to CSV format

Create report for NIST Tech Pubs database (fetches from GitHub)

The purpose of this command is to aid the NIST PubID team in assessing the impact and type of changes to be made in enacting this new scheme.

By default, the report command generates a table to indicate which migrated identifiers have changed, focusing on changes of two styles:

PubID in short style vs legacy publication ID
PubID in machine-readable style vs legacy DOI

As seen in the following output, a ✅ or a - will be shown in the appropriate column of change.

$ pubid-nist report
ID changed? | New PubID | Document ID | DOI changed? | New PubID-MR | DOI | Title
 - | NBS BH 1 | NBS BH 1 |  - | NBS.BH.1 | NBS.BH.1 | Recommended minimum requirements for small dwelling construction : report of Building Code Committee July 20, 1922
 - | NBS BH 10 | NBS BH 10 |  - | NBS.BH.10 | NBS.BH.10 | A city planning primer by the advisory committee on zoning appointed by Secretary Hoover
 ...
✅ | NBS BH 3A | NBS BH 3a | ✅ | NBS.BH.3A | NBS.BH.3a | A zoning primer by the advisory committee on zoning appointed by Secretary Hoover (Revised)
 - | NBS BH 4 | NBS BH 4 |  - | NBS.BH.4 | NBS.BH.4 | How to own your home : a handbook for prospective home owners
✅ | NBS BH 5A | NBS BH 5a | ✅ | NBS.BH.5A | NBS.BH.5a | A standard state zoning enabling act under which municipalities may adopt zoning regulations by the advisory committee on zoning appointed by Secretary Hoover (revised edition 1926)
...
✅ | NBS RPT 2751 | NBS report ; 2751 |  - | NBS.RPT.2751 | NBS.RPT.2751 | Stochastic search for the maximum of a function
 ...
✅ | NBS RPT 2831 | NBS report ; 2831 |  - | NBS.RPT.2831 | NBS.RPT.2831 | Error bounds for eigenvalues of symmetric integral equations

Better yet, the report command supports CSV output. The "changes" fields will display true or false accordingly.

$ pubid-nist report --csv
ID changed?,New PubID,Document ID,DOI changed?,New PubID-MR,DOI,Title
false,NBS BH 1,NBS BH 1,false,NBS.BH.1,NBS.BH.1,"Recommended minimum requirements for small dwelling construction : report of Building Code Committee July 20, 1922"
false,NBS BH 10,NBS BH 10,false,NBS.BH.10,NBS.BH.10,A city planning primer by the advisory committee on zoning appointed by Secretary Hoover
false,NBS BH 11,NBS BH 11,false,NBS.BH.11,NBS.BH.11,A standard city planning enabling act by the advisory committee on city planning and zoning appointed by secretary Hoover
...
true,NIST SP 260-214,NIST SP 260-14,false,NIST.SP.260-214,NIST.SP.260-214,"Analysis of Seafood Reference Materials: RM 8256, RM 8257, RM 8258 and RM 8259, Wild-Caught Coho Salmon (RM 8256), Aquacultured Coho Salmon (RM 8257), Wild-Caught Shrimp (RM 8258), Aquacultured Shrimp (RM 8259)"
false,NIST SP 260-14,NIST SP 260-14,false,NIST.SP.260-14,NIST.SP.260-14,"Analysis of Seafood Reference Materials: RM 8256, RM 8257, RM 8258 and RM 8259, Wild-Caught Coho Salmon (RM 8256), Aquacultured Coho Salmon (RM 8257), Wild-Caught Shrimp (RM 8258), Aquacultured Shrimp (RM 8259)"
true,NIST IR 8379,NISTIR 8379,false,NIST.IR.8379,NIST.IR.8379,Summary Report for the Virtual Workshop Addressing Public Comment on NIST Cybersecurity for IoT Guidance

The best part is that this CSV will work properly with spreadsheet editors like Excel and Pages. All you need is to export the CSV values to a CSV file, and open it in your favorite program.

$ pubid-nist report --csv > myreport.csv

It is easy to filter these columns in Microsoft Excel with the following steps:

Open the CSV file in Excel
Convert the header row into a filter row: first highlight the header row, then click on "Data > Filter"
Filter the columns accordingly

Figure 1. PubID conversion report, showing mapping between legacy publication identifiers and NIST PubIDs

Now it’s easy to screen through the anomalies and surface the data issues!

Elements of the PubID

Publisher

Name	Abbrev	Short
National Institute of Standards and Technology	Natl. Inst. Stand. Technol.	NIST
National Bureau of Standards	Natl. Bur. Stand.	NBS

Series

Publisher	Prefix	Name	Abbrev	MR (with Publisher)	Example
NIST	NIST AMS	Advanced Manufacturing Standard	Adv. Man. Ser	`NIST.AMS`	NIST AMS 200-2
NIST	NIST BSS	Building Science Series	Bldg. Sci. Ser.	`NIST.BSS`	NIST BSS 181
NBS	NBS BSS	Building Science Series	Bldg. Sci. Ser.	`NBS.BSS`	NBS BSS 94
NBS	NBS BMS	Building Material Structures Report	N/A	`NBS.BMS`	NBS BMS 140 Ed. 2
NBS	NBS BRPD-CRPL-D	Basic Radio Propagation Predictions Series	N/A	`NBS.BRPD-CRPL-D`	NBS BRPD-CRPL-D 209
NBS	NBS BH	Building and Housing Reports	N/A	`NBS.BH`	NBS BH 18
NBS	NBS CRPL	Central Radio Propagation Laboratory Reports	N/A	`NBS.CRPL`	NBS CRPL 6-3
NBS	NBS CRPL-F-A	CRPL Ionospheric Data	N/A	`NBS.CRPL-F-A`	NBS CRPL-F-A 245
NBS	NBS CRPL-F-B	CRPL Solar-Geophysical Data	N/A	`NBS CRPL-F-B245`	NBS CRPL-F-B245
NBS	NBS IP	CRPL Ionospheric Predictions	N/A	`NBS.IP`	NBS IP 25
NBS	NBS CIRC	Circulars	N/A	`NBS.CIRC`	NBS CIRC 460sup1962
NBS	NBS CIS	Consumer Information Series	N/A	`NBS.CIS`	NBS CIS 10
NBS	NBS CS	Commercial Standards	N/A	`NBS.CS`	NBS CS 113-51
NBS	NBS CSM	Commercial Standards Monthly	N/A	`NBS.CSM`	NBS CSM v9n10
NIST	FIPS PUB	Federal Information Processing Standards Publication	Federal Inf. Process. Stds.	`NIST.FIPS`	FIPS PUB 202
NIST	NISTGCR	Grant/Contract Reports	N/A	`NIST.GCR`	NIST GCR 17-917-45
NBS	NBS GCR	Grant/Contract Reports	N/A	`NBS.GCR`	NBS GCR 77-82
NIST	NIST HB	Handbook	Handb.	`NIST.HB`	NIST Handbook 150-872
NBS	NBS HB	Handbook	Handb.	`NBS.HB`	NBS Handbook 137
NBS	NBS HR	Hydraulic Research in the United States	N/A	`NBS.HR`	NBS HR 14A
NBS	NBS IRPL	Interservice Radio Propagation Laboratory	N/A	`NBS.IRPL`	NBS IRPL 27
NIST	ITL Bulletin	ITL Bulletin	N/A	`NIST.ITLB`	NIST ITL Bulletin August 2020
NIST	NIST LC	Letter Circular	N/A	`NIST.LC`	NIST LC 1136
NBS	NBS LC	Letter Circular	N/A	`NBS.LC`	NBS LC 1128
NIST	NIST MN	Monograph	Monogr.	`NIST.MN`	NIST Monograph 175
NBS	NBS MN	Monograph	Monogr.	`NBS.MN`	NIST Monograph 125, NIST Monograph 125, Supp. 1
NBS	NBS MP	Miscellaneous Publications	N/A	`NBS.MP`	NBS MP 260e1968
NIST	NIST NCSTAR	National Construction Safety Team Report	Natl. Constr. Tm. Act Rpt.	`NIST.NCSTAR`	NIST NCSTAR 1-1A
NIST	NIST NSRDS	National Standard Reference Data Series	Natl. Stand. Ret. Data Ser.	`NIST.NSRDS`	NIST NSRDS 100-2021
NBS	NSRDS-NBS	National Standard Reference Data Series	Natl. Stand. Ret. Data Ser.	`NBS.NSRDS`	NSRDS-NBS 1
NIST	NIST IR	Interagency or Internal Report	N/A	`NIST.IR`	NISTIR 8347
NBS	NBS IR	Interagency or Internal Report	N/A	`NBS.IR`	NBSIR 79-1776
NIST	NIST OWMWP	Office of Weights and Measures White Papers	N/A	`NIST.OWMWP`	NIST OWMWP 06-13-2018
NBS	NBS PC	Photographic Circulars	N/A	`NBS.PC`	NBS RPT 10394
NBS	NBS RPT	Reports	N/A	`NBS.RPT`	NBS PC 1
NIST	NIST PS	Voluntary Product Standards	Prod. Stand.	`NIST.PS`	NIST PS 20-20
NBS	NBS SIBS	Special Interior Ballistics Studies	N/A	`NBS.SIBS`	NBS SIBS 1
NBS	NBS PS	Voluntary Product Standards	Prod. Stand.	`NBS.PS`	NBS PS 15-69
NIST	NIST SP	Special Publication	Spec. Publ.	`NIST.SP`	NIST SP 800-115
NBS	NBS SP	Special Publication	Spec. Publ.	`NBS.SP`	NBS SP 500-137
NIST	NIST TN	Technical Note	Tech. Note	`NIST.TN`	NIST TN 2156
NBS	NBS TN	Technical Note	Tech. Note	`NBS.TN`	NBS TN 876
NBS	NBS TIBM	Technical Information on Building Materials	N/A	`NBS.TIBM`	NBS TIBM 61
NIST	NIST TTB	Technology Transfer Brief	N/A	`NIST.TTB`	NIST TTB 2
NIST	NIST DCI	Data Collection Instruments	Data Collect. Instr.	`NIST.DCI`	NIST DCI 002
NIST	NIST EAB	Economic Analysis Brief	N/A	`NIST.EAB`	NIST EAB 3
NIST	NIST Other	Other	Other	`NIST.O`	Report to the President
NIST	CSRC White Paper	Cybersecurity Resource Center White Paper	CSWP	`NIST.CSWP`	NIST.CSWP.04282021
NIST	CSRC Book	Cybersecurity Resource Center Book	CSRC Book	`NIST.CSB`	Executive Guide to Computer Security, Metrics to Security
NIST	CSRC Use Case	Cybersecurity Resource Center Use Case	CSRC Use Case	`NIST.CSUC`	Wireless Medical Infusion Pumps: Medical Device Security
NIST	CSRC Building Block	Cybersecurity Resource Center Building Block	CSRC Building Block	`NIST.CSBB`	Domain Name System-Based Security for Electronic Mail
NIST	JPCRD	Journal of Physical and Chemical Reference Data	J. Phys. & Chem. Ref. Data	`JPCRD`	(excluded from PubID scheme)
NIST	JRES	Journal of Research of NIST	J. Res. Natl. Inst. Stan.	`NIST.JRES`	(excluded from PubID scheme)

Stage

The stage code element only applies to non-final publications.

In most series, documents are only released as final publications, and therefore their PubIDs will not contain a stage code.

Only some series support stage codes, e.g. SP 800 and SP 1800.

Name	Value
Initial Public Draft	IPD
Second Public Draft (to the Nth Public Draft)	2PD (… nPD)
Final Public Draft	FPD
Work-in-Progress Draft	WD
Preliminary Draft	PreD

Report number

The contents and pattern of the report number are dependent on the series.

Possible values:

{sequence number}
{subseries}-{sequence number}
{sequence number}-{volume}
{sequence number}-{edition}
{subseries}-{sequence number}-{edition}
etc.

A "Part" can also be indicated by an appended alphabetic character to the end.

Part

Name	Abbrev and Short	MR
Part	Pt.	`pt`
Volume	Vol.	`v`
Section	Sec.	`sec`
Supplement	Suppl.	`sup`
Index	Index	`indx`
Addendum	Add.	`add` (TBC with NIST)
Insert	Ins.	`ins` (TBC with NIST)
Errata	Err.	`err` (TBC with NIST)
Appendix	App.	`app` (TBC with NIST)

Edition

Name	Abbrev and Short	MR
Revision	Rev.	`r`
Edition	Ed.	`e`
Version	Ver.	`ver`

Translation

An ISO 639-2 code that represents a translated document from English.

If a document is translated from English, suffix the document with a 3-letter ISO 639-2 code within parentheses.

Raw values seen in legacy DOIs include:

Name	Correct value	MR	Legacy values seen in DOI
Spanish	(ESP)	`esp`	`es`
Vietnamese	(VIE)	`vie`	`viet`
Portuguese	(POR)	`por`	`port` `pt`
Chinese	(ZHO)	`zho`	`chi`
Indonesian	(IND)	`ind`	`id`

Update

General

When a document is updated with an errata, the original edition may be reissued to include the errata.

These documents will display the text "includes updates as of…".

In this case the document identifier will include the element "Update".

Name	Abbrev and Short	MR
Update	Upd.	`u`

Note	This is similar to the ISO/IEC "Amd." and "Cor." except that the NIST "Upd." is applied to the original document and (typically) does not exist separately.

Update number

A sequential integer numbering of the update counting from the original document.

The first update is numbered 1, and so forth.

Update year

The year last updated, shown as a suffix to the identifier.

"{identifier}:{update-year}"

PubID patterns

Presentation

Generally in this order:

No update: {series} {stage} {report number}{part}{edition}({translation})
With update: {series} {stage} {report number}{part}{edition}({translation})/{update} {update number}:{update year}

Full PubID

Samples:

National Institute of Standards and Technology Federal Information Processing Standards Publication 199
National Institute of Standards and Technology Special Publication 800-27, Revision A
National Institute of Standards and Technology Special Publication 800-39 (Second Public Draft)

(originally described in https://github.com/metanorma/metanorma-nist/issues/98)

Abbreviated PubID

This form is used in the Authority section.

Natl. Inst. Stand. Technol. {abbrev(series)} {reportnumber} {part | abbrev(volume)}, {abbrev(revision)}

abbrev(series) represent the abbreviation of the Series title
(Note: there is no {update-date})

Examples:

"Natl. Inst. Stand. Technol. Spec. Publ. 800-78-4"
"Natl. Inst. Stand. Technol. Spec. Publ. 800-116"
"Natl. Inst. Stand. Technol. Spec. Publ. 800-57 Part 1, Revision 4"

(originally from https://github.com/metanorma/metanorma-nist/issues/88)

Short PubID

The "short form" is used to cite the documents within text.

It is used in these situations:

Locality references. "In Section 3.2 of SP 800-187…" (the "SP 800-187" is a link).

Note
NIST pubs are composed of "Sections" not "Clauses"
A generic document reference. "SP 800-53 describes…". This form does not specify a revision or update date.
"All parts". "The SP 800-57 subseries describes key management…".

("NISTIR 6885 2003 Edition (February 2003)" is a poor indicator.)

The correct rule should be:

{abbrev(series)} {reportnumber} {abbrev(volume)} {abbrev(revision)} {edition, optional}

Note	For FIPS, `reportnumber` is the full FIPS number, including revision, e.g., `140-2`.

(originally from https://github.com/metanorma/metanorma-nist/issues/88)

Short form date:

Month YYYY

Examples:

NIST SP 800-53r4 (20152201) supersedes NIST SP 800-53r4 (20140115)
NIST SP 800 63A (December 2017) supersedes NIST SP 800-63A
NIST SP 800 57 Part 1 Revision 4 supersedes NIST SP 800-57 Part 1 Revision 3 ("Rev." is also accepted, and converted to "Revision")
NIST SP 800 160 Volume 1 supersedes NIST SP 800-160 (20180103) ("Vol." is also accepted, and converted to "Volume")
Undated form "NIST SP 800 53r4"

Strip Revision and Date from title, only if the Revision and Date are unique for each document number. These are identified as "Rev. …", "Revision …." and " (Month YYYY)", whichever comes first.

Machine-readable PubID

Machine-readable syntax:

{publisher}.{series}.[{stage}.]{reportnumber}.{part}.{revision}.[{lang}.]{update-date}

{publisher}.{series}.[{stage}.]{reportnumber}.{part}.{revision}.[{lang}.][{update}]{update-date}

Generally, this rule should be able to uniquely identify an edition of a document.

{part}
- Part
  - A "Part 1" document is encoded as "pt1";
  - When a letter part is indicated, e.g. "800-63A", we should keep it as part of the reportnumber (i.e., don’t indicate "pt").
- Volumes
  - "Volume 1" is encoded as "v1";
{revision}
- "Revision 1" is encoded as "r1"
- If a superseding edition is a full revision, it will get the next Rev. #.
- If a superseding edition is just an errata update, we use the update date from the title page ("includes updates as of …") to uniquely identify this edition. Preferably in the -yyyymmdd format.
{update}
- "Update 1" is encoded as "upd1"

Some examples:

NIST.SP.800-53r4-20152201 supersedes NIST.SP.800-53r4-20140115
NIST.SP.800-63A-20171201 supersedes NIST.SP.800-63A
NIST.SP.800-57pt1r4 supersedes NIST.SP.800-57pt1r3
NIST.SP.800-160v1 supersedes NIST.SP.800-160-20180103
NIST.IR.8204.upd1-2019 supersedes NIST.IR.8204
The undated form is NIST.SP.800-53r4

(originally from https://github.com/metanorma/metanorma-nist/issues/88)

Examples of PubID

SP 800 series
- NIST SP 800-40r3
- NIST SP 800-45ver2
- NIST SP 800-53r4/Upd 3:2015
- NIST SP IPD 800-53r5
- NIST SP 800-53Ar4/Upd 1:2014
- NIST SP 800-60v1r1
- NIST SP 800-57pt1r4
- NIST SP 800-73-4/Upd 1:2016
- NIST SP 2PD 800-188
SP 1800 series
- NIST SP 2PD 1800-13B
- NIST SP PreD 1800-19B
NIST IR series
- NIST IR 8011v3
- NIST IR 8204/Upd 1:2019
- NIST IR 8115(spa)
Other NIST Series
- NIST HB 130e2019
- NIST SP 1041r1/Upd 1:2012
- NIST NCSTAR 1-1Cv1

pubid-nist's People

Contributors

Stargazers

Watchers

Forkers

mico isimluk

pubid-nist's Issues

(URGENT) Conversion error: "NIST SP 800-53r4" should output "NIST SP 800-53r4"

$ bundle exec nist-pubid convert "NIST SP 800-53r4"
NIST SP 800-53-4

This is wrong. As previously described, the revision element should always default to use "rN" instead of "-N".

(URGENT) Incorrect conversion for "NBS CIRC 25insert"

Right now it converts as:

$ nist-pubid convert "NBS CIRC 25insert"
NBS CIRC 25INSERT

But it should be NBS CIRC 25ins.

(URGENT) Parse failure: Unable to handle CSWP documents

bundle exec nist-pubid convert "NIST CSWP 04282021"
bundler: failed to load command: nist-pubid (/Users/me/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/bin/nist-pubid)
/Users/me/src/mn/nist-pubid/lib/nist_pubid/document.rb:144:in `parse': failed to parse serie for NIST CSWP 04282021 (NistPubid::Errors::ParseError)
	from /Users/me/src/mn/nist-pubid/exe/nist-pubid:44:in `convert'

Validate implementation with official NIST Tech Pubs collection

https://github.com/usnistgov/NIST-Tech-Pubs/blob/nist-pages/xml/allrecords.xml

Data e.g.

<body>
   <query key="BH">
      <doi type="report-paper_title">10.6028/NBS.BH.1</doi>
      <crm-item name="publisher-name" type="string">National Institute of Standards and Technology (NIST)</crm-item>
      <crm-item name="prefix-name" type="string">National Institute of Standards and Technology</crm-item>
      <crm-item name="member-id" type="number">4068</crm-item>
      <crm-item name="citation-id" type="number">78696207</crm-item>
      <crm-item name="book-id" type="number">2050209</crm-item>
      <crm-item name="deposit-timestamp" type="number">201511031134</crm-item>
      <crm-item name="owner-prefix" type="string">10.6028</crm-item>
      <crm-item name="last-update" type="date">2018-03-06T09:55:24Z</crm-item>
      <crm-item name="created" type="date">2015-11-04T17:31:05Z</crm-item>
      <crm-item name="citedby-count" type="number">0</crm-item>
      <doi_record>
         <report-paper>
            <report-paper_metadata language="en">
               <contributors>
                  <person_name sequence="first" contributor_role="author">
                     <given_name>Ira H</given_name>
                     <surname>Woolson</surname>
                  </person_name>
<!-- ... -->
               </contributors>
               <titles>
                  <title>Recommended minimum requirements for small dwelling construction :</title>
                  <subtitle>report of Building Code Committee July 20, 1922</subtitle>
               </titles>
               <edition_number>0</edition_number>
               <publication_date media_type="online">
                  <year>1923</year>
               </publication_date>
               <publisher>
                  <publisher_name>National Bureau of Standards</publisher_name>
                  <publisher_place>Gaithersburg, MD</publisher_place>
               </publisher>
               <institution>
                  <institution_name>National Bureau of Standards</institution_name>
                  <institution_acronym>NBS</institution_acronym>
                  <institution_place>Gaithersburg, MD</institution_place>
               </institution>
               <publisher_item>
                  <item_number item_number_type="report-number">NBS BH 1</item_number>
               </publisher_item>
               <doi_data>
                  <doi>10.6028/NBS.BH.1</doi>
                  <resource>https://nvlpubs.nist.gov/nistpubs/Legacy/BH/nbsbuildinghousing1.pdf</resource>
               </doi_data>
            </report-paper_metadata>
         </report-paper>
      </doi_record>
   </query>

We want to be able to generate a PubID for every entry, using a combination of:

<item_number>
<edition_number>
<publication_date> <year/month>
<doi>

Right now that information is handled in https://github.com/relaton/relaton-nist/blob/master/lib/relaton_nist/nist_bibliography.rb

And create a map.

NIST PubID scheme: Support document type "Appendix"

The 2021-08 scheme does not support the part type "Appendix".

"NIST SP 250e1992app"

https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication250e1992app.pdf

CLI should fail gracefully on invalid identifier

$ nist-pubid convert "NBS LCIRC 10"
Calling `DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has been deprecated. Please call `DidYouMean.correct_error(error_name, spell_checker)' instead.
bundler: failed to load command: nist-pubid (/Users/me/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/bin/nist-pubid)
/Users/me/src/mn/nist-pubid/lib/nist_pubid/document.rb:195:in `parse_docnumber': failed to parse document identifier for NBS LCIRC 10 (NistPubid::Errors::ParseError)
	from /Users/me/src/mn/nist-pubid/lib/nist_pubid/document.rb:157:in `parse'
	from /Users/me/src/mn/nist-pubid/exe/nist-pubid:44:in `convert'
	from /Users/me/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/gems/thor-1.0.1/lib/thor/command.rb:27:in `run'
...

The CLI should catch this error and fail with an error message that:

[Error] This does not seem to be a valid NIST Tech Pubs legacy identifier

Support abbreviated and long parameters for PubID parsing

While working on integration to relaton-nist I found it's using PubID with additional parameters in long representation:
bib = RelatonNist::NistBibliography.get "NIST IR 8011 Vol. 4"
Along with short version:
bib = RelatonNist::NistBibliography.get "NIST IR 8011v4"

Looking at proposal (https://www.nist.gov/system/files/documents/2021/08/26/Publication-ID-Proposal_26Aug21.pdf), I found there is also different options how authors could refer to publications:

• NIST SP 800-53, Revision 4, includes updates as of 01-22-2015
• NIST SP 800-53 (April 2013), updated January 22, 2015
• NIST SP 800-53, Rev. 4, updated 01-22-2015

They are references to the same document.

Now we only parsing using short notation (NIST SP 800-53r4).

Should we add support to parse using abbreviated and long PubID parameters?

NIST data source clarification: Duplicated IDs in NIST source

Originally posted here relaton/relaton-nist#53 (comment)
There are item number duplicates in the NIST source
These item numbers were found 2 or more times:

["NBS CIRC 46e2",
 "NIST HB 105-1-1990",
 "NBS HB 67suppJune1965",
 "NIST IR 89-4220",
 "NBS TN 789-1",
 "NIST HB 150-10",
 "NIST IR 8115",
 "NIST IR 8117",
 "NIST IR 8119",
 "NIST IR 8178",
 "NIST TN 1648"]

For example NBS.CIRC.36e2 and NBS.CIRC.46e2 have NBS CIRC 46e2 item numbers, which looks like a mistake.

Add CLI interface for testing / sanity checking

Two use cases:

From the current allrecords.xml, get a table of '| current ID | DOI | Title | final PubID |" so they can visually check. And it would be great if we can generate a CSV so the PubID team can load it up in Excel etc.
When provided with a ficticious/example old publication ID, show the new PubID

NIST PubID scheme: translation identifier in MR style

Right now we have these:

PubID MR	DOI
NIST.IR.8115(zho)	NIST.IR.8115chi
NIST.IR.8115(spa)	NIST.IR.8115es
NIST.IR.8115(vie)	NIST.IR.8115viet
NIST.IR.8117(spa)	NIST.IR.8117es
NIST.IR.8118r1(spa)	NIST.IR.8118r1es
NIST.IR.8119(zho)	NIST.IR.8119chi
NIST.IR.8119(spa)	NIST.IR.8119es
NIST.IR.8119(vie)	NIST.IR.8119viet
NIST.IR.8178(por)	NIST.IR.8178port

We cannot have parentheses in the MR.

Should we use a dot separator, e.g. NIST.IR.8115.zho, or some other separator?

NIST PubID scheme: Support document type "Insert"

The document type "insert" is not supported by the 2021-08 scheme.

e.g. "NBS CIRC 25insert"
https://nvlpubs.nist.gov/nistpubs/Legacy/circ/nbscircular25insert.pdf

Adopt GHA auto-release workflow

Just like this:

https://github.com/metanorma/mn2pdf-ruby/blob/main/.github/workflows/release.yml

The variable names for secrets are identical.

(URGENT) "NIST IR 6806rNNNN", the "NNNN" is edition not part

NIST IR 6806-2002	NIST IR 6806r2002
NIST IR 6806-2004	NIST IR 6806r2004

Support document type "Supplement"

NIST has the "Supplement" document type and also the Addendum (https://github.com/metanorma/metanorma-nist/issues/259).

"Supplement" is a numbered attribute like Addendum:

"Supplement 1" is just called "Supplement"
"Supplement 2" is just called "Supplement 2"

Examples:

Representation: e.g.

In PubID, the "Supplement":

short: "Suppl. X"
long: "Supplement X"

Originally posted by @ronaldtse in #1 (comment)

(URGENT) FIPS PUB losing date of publication/edition

PubID	Original
NBS FIPS PUB 1-2	NBS FIPS 1-2-1974
NBS FIPS PUB 1-2	NBS FIPS 1-2-1977
NBS FIPS PUB 1-2	NBS FIPS 1-2-1984

These are documents of different editions. We lost the identification here.

Move commented document identifiers to specs

https://github.com/metanorma/nist-pubid/blob/14cc82a9c3d910e0416a5d4de887420e19302730/spec/nist_pubid/document_spec.rb#L3-L21

Provide review comments for newly published NIST PubID draft document

From NIST:

We have released a news item on NIST.gov that includes a link to our Publication identifier proposal, and a request for comments by September 30. Please share with anyone in your networks that would be interested. And, as always, please let us know if you have any questions.

https://www.nist.gov/news-events/news/2021/08/nist-technical-series-publications-proposed-publication-identifier-syntax

direct link to the proposal: https://www.nist.gov/system/files/documents/2021/08/26/Publication-ID-Proposal_26Aug21.pdf

@andrew2net can you check if we are missing anything in the nist-pubid gem and Relaton NIST?
@opoudjis can you review to see if we have any gaps in document IDs right now?
@strogonoff @wkwong-ribose @ribose-jeffreylau please have a look.

We will need to provide comments by September 30, so please help provide them in this thread for compilation by mid September.

Parse NBS DOIs and support NBS documents in the conversion to PubIDs

NBS is the previous name of NIST, and they may have slightly different identifier patterns.

From @andrew2net :

"NBS.BMS.140e2" is the "e2" an edition? Does it map to "Rev. 2" or "Ver. 2" in PubID?

"e2" is Edition. Edition is separate from Revision and Version.

We should not change the NBS name into NIST PubID, but keep the original pattern. We need to discover these patterns.

Parsing issues: "parse error" elements need to be fixed

Seems that many of these are data issues, we will need clarification from NIST.

ID changed?	New PubID	Document ID	DOI changed?	New PubID-MR	DOI
TRUE	parse error	NBS CIRC e2	TRUE	parse_error	NBS.CIRC.e2
TRUE	parse error	NBS CIRC sup	TRUE	parse_error	NBS.CIRC.sup
TRUE	parse error	NBS CIRC supJun1925-Jun1926	TRUE	parse_error	NBS.CIRC.supJun1925-Jun1926
TRUE	parse error	NBS CIRC supJun1925-Jun1927	TRUE	parse_error	NBS.CIRC.supJun1925-Jun1927
TRUE	parse error	NBS CRPL c4-4	TRUE	parse_error	NBS.CRPL.c4-4
TRUE	parse error	NBS CS e104-43	TRUE	parse_error	NBS.CS.e104-43
TRUE	parse error	NBS CS e106-43	TRUE	parse_error	NBS.CS.e106-43
TRUE	parse error	NBS CS e107-43	TRUE	parse_error	NBS.CS.e107-43
TRUE	parse error	NBS CS e119-45	TRUE	parse_error	NBS.CS.e119-45
TRUE	parse error	NBS CS e124-45	TRUE	parse_error	NBS.CS.e124-45
TRUE	parse error	NBS CS e15-43	TRUE	parse_error	NBS.CS.e15-43
TRUE	parse error	NBS RPT ADHOC	TRUE	parse_error	NBS.RPT.ADHOC
TRUE	parse error	NBS report ; Apr-Jun1948	TRUE	parse_error	NBS.RPT.Apr-Jun1948
TRUE	parse error	NBS report ; Apr-Jun1949	TRUE	parse_error	NBS.RPT.Apr-Jun1949
TRUE	parse error	NBS report ; Apr-Jun1950	TRUE	parse_error	NBS.RPT.Apr-Jun1950
TRUE	parse error	NBS report ; Apr-Jun1951	TRUE	parse_error	NBS.RPT.Apr-Jun1951
TRUE	parse error	NBS report ; div9	TRUE	parse_error	NBS.RPT.div9
TRUE	parse error	NBS report ; Jan-Jun1971	TRUE	parse_error	NBS.RPT.Jan-Jun1971
TRUE	parse error	NBS report ; Jan-Mar1948	TRUE	parse_error	NBS.RPT.Jan-Mar1948
TRUE	parse error	NBS report ; Jan-Mar1950	TRUE	parse_error	NBS.RPT.Jan-Mar1950
TRUE	parse error	NBS report ; Jul-Dec1947	TRUE	parse_error	NBS.RPT.Jul-Dec1947
TRUE	parse error	NBS report ; Jul-Sep1949	TRUE	parse_error	NBS.RPT.Jul-Sep1949
TRUE	parse error	NBS report ; Jul-Sep1950	TRUE	parse_error	NBS.RPT.Jul-Sep1950
TRUE	parse error	NBS report ; Oct-Dec1949	TRUE	parse_error	NBS.RPT.Oct-Dec1949
TRUE	parse error	NBS report ; Oct-Dec1950	TRUE	parse_error	NBS.RPT.Oct-Dec1950

These are already reported in:

NIST PubID scheme: Support document type "Addendum"

"NIST SP 800-38a-add" is an Addendum.

The "addendum" type should be added to the {part} portion of the 2021-08 scheme.

(URGENT) Broken parsing for "NISTGCR 20-026"

NISTGCR 20-026	NIST GCR 20-026	TRUE	NISTGCR.20-026	NIST.GCR.20-026

This is supposed to be "NIST GCR 20-026".

NIST PubID scheme: identifiers that need clarification

A number of Tech Pubs have current pub IDs that contain both Edition/Revision and also Year. Some of them have Month information.

What should the patterns be for those? I’m listing them below where the left shows the original pub ID, the right shows a proposed PubID.

What do we do with the month when given? (still an issue in PubID 1.0)

"NBS CIRC 13e2revjune1908”
"NBS FIPS 89-May1981”
"NIST IR 8170 (March 2020)"

Should we omit the year information if there is an edition number already provided? (still an issue in PubID 1.0)

“NBS CIRC 11e2-1915” => “NBS CIRC 11e2"
“NBS HB 44e2-1955” => “NBS HB 44e2”?

Should we omit the edition number if it is 1? (still an issue in PubID 1.0)

“NBS SP 960-12e1” => “NBS SP 960"
~~"NIST SP 260-162 2006ed.” => "NIST SP 260-162” (this is Edition 1 of SP 260-162)~~ (now at #155)
~~“NIST SP 250-29e2004” => “NIST SP 250-29"~~ (now at #155)

Should we omit the revision number if it was mis-marked? (still an issue in PubID 1.0)

"NIST SP 260-174r2012” => "NIST SP 260-174” (this is not a revision, this is actually Edition 1)
"NIST SP 305-23r1992” => "NIST SP 305-23” (this is actually Edition 1)

Should we automatically assign the edition number if the edition is given as a year? (No, an edition can be yyyy or 1-9, described in #155)

“NIST SP 250-39e2009” => “NIST SP 250-39e2” (this is Edition 2 of SP 250-39)

Sometimes revision numbers are inconsistently assigned: (still an issue in PubID 1.0, source data needs to be fixed)

"NIST SP 800-57p1” is the base document.
"NIST SP 800-57p1r2006” => should have been r1
"NIST SP 800-57p1r2007” => should have been r2
"NIST SP 800-57p1r3” => is correct.

Should the Part number, if in alphabet, be capitalised? (now decided in #154)

“NIST SP 800-131a” => “NIST SP 800-131A”? (The base document “NIST SP 800-131a” has a R1 as “NIST SP 800-131Ar1”.)
"NIST SP 800-38a” => "NIST SP 800-38A”? ("NIST SP 800-38a” but "NIST SP 800-38F”)

Should PubID fix minor issues like these? (still an issue in PubID 1.0, source data needs to be fixed)

"NIST SP 800-40ver2” => "NIST SP 800-40v2”

“Part” representation sometimes goes with “p” or “pt”: (decided in #87)

"NIST SP 800-57pt1r4” vs "NIST SP 800-57p3"

There are also some funny ones that need consideration: (still an issue in PubID 1.0, potentially data source issues)

“NIST SP 800-27 Revision A”
“NIST SP 800-22 Revision 1a”
"NIST SP 800-87ver1” vs "NIST SP 800-87ver1e2006” vs "NIST SP 800-87ver1e2007”

NIST PubID scheme: Confirm that "NISTGCR" is to be shown as "NIST GCR" in PubID

As per #57 .

NIST PubID scheme: Support document type "Errata"

The Errata document type is not supported in {part}, need to add here.

This is a Revision 1 of AMS 300-8. The parsed PubID now puts the "1" as a "part" which is wrong.
After looking at these examples in https://www.nist.gov/system/files/documents/2021/08/26/Publication-ID-Proposal_26Aug21.pdf I made it work this way.

I got to conclusion: If there are no other parts (volume, part, edition) then PubID with revision should look like: `NIST AMS 300-8-1`. But if there are other part, then revision with letter will be added, example: `NIST AMS 300-8v1r1`

Interesting. Did not know sometimes the edition/revision is represented as "-N". I think it will be very confusing to use "-N" as edition vs "rN" (e.g revision 4 as "r4"). We will have to report back to NIST that this is an issue.

Originally posted by @mico in #8 (comment)

(URGENT) FIPS PUBS MR scheme wrong (`NIST.FIPS` not `NIST.FIPS.PUB`)

See https://github.com/metanorma/nist-pubid/blob/main/README.adoc

See these:

MR	DOI
NBS.FIPS.PUB.34	NBS.FIPS.34
NBS.FIPS.PUB.35	NBS.FIPS.35
NBS.FIPS.PUB.36	NBS.FIPS.36
NBS.FIPS.PUB.36-1975	NBS.FIPS.36-1975

NIST data source PubID: some NCSTAR documents do not use Volume explicitly but are named as such

In the NIST PubID, "vN" means "Volume N". Are the following correct?

"NIST NCSTAR 1-5av1" is "NIST NCSTAR 1-5A Chapters 1-8".

"NIST NCSTAR 1-5av2" is "NIST NCSTAR 1-5A Chapter 9 - Appendix C".

"NIST NCSTAR 1-5av3" is "NIST NCSTAR 1-5A Appendices D-G".

"NIST NCSTAR 1-5av4" is "NIST NCSTAR 1-5A Appendices H-M".

"NIST NCSTAR 1-5bv1" is actually "NIST NCSTAR 1-5B". (includes Chapter 1 to Appendix G, i.e. fully covers "NIST NCSTAR 1-5bv2")

"NIST NCSTAR 1-5bv2" is "NIST NCSTAR 1-5B Appendices D-G" (a subset of "NIST NCSTAR 1-5bv1").

"NIST NCSTAR 1-6dv1" is "NIST NCSTAR 1-6D Chapters 1-3".

"NIST NCSTAR 1-6dv2" is "NIST NCSTAR 1-6D Chapter 4 - Appendix C"

"NIST NCSTAR 1-6v1" is "NIST NCSTAR 1-6 Chapters 1-6".

"NIST NCSTAR 1-6v2" is "NIST NCSTAR 1-6 Chapter 7 - Appendix B".

"NIST NCSTAR 1-8v1" is "NIST NCSTAR 1-8" (actually it contains Chapters 1-9, no Appendices, but with "Attachments").

"NIST NCSTAR 1-8v2" is "NIST NCSTAR 1-8 (Appendices A-I)".

"NIST NCSTAR 1-8v3" is "NIST NCSTAR 1-8 (Appendices J-L)".

"NIST NCSTAR 2v1" is "NIST NCSTAR 2: Volume 1". So "NIST NCSTAR 2v1" is accurate.

"NIST NCSTAR 3" is correct.

Update PubID scheme to support all NIST Tech Pubs series

All series from here:
https://pages.nist.gov/NIST-Tech-Pubs/series.html

Advanced Manufacturing Series
Basic Radio Propagation Predictions Series
Building and Housing Reports
Building Materials and Structures Reports
Building Science Series
Central Radio Propagation Laboratory Reports
CRPL Ionospheric Data
CRPL Ionospheric Predictions
CRPL Solar-Geophysical Data
Circulars
Consumer Information Series
Commercial Standards
Commercial Standards Monthly
Cybersecurity White Papers
Economic Analysis Briefs
Federal Information Processing Standards Publications
Grant/Contractor Reports
Handbooks
Hydraulic Research in the United States
Interservice Radio Propagation Laboratory
Letter Circular
Monographs
Miscellaneous Publications
National Construction Safety Team Act Reports
National Standard Reference Data Series
NISTIRs (Interagency/Internal Reports)
Office of Weights and Measures White Papers
Photographic Circulars
NBS Reports
Special Interior Ballistics Studies
Special Publications (General)
SP 250: Calibration Services
SP 260: Standard Reference Materials
SP 300: Precision Measurement and Calibration
SP 400: Semiconductor Measurement Technology
SP 480: Law Enforcement Technology
SP 500: Computer Systems Technology
SP 700: Industrial Measurement Series
SP 800: Computer Security Series
SP 823: Integrated Services Digital Network Series
SP 960: NIST Recommended Practice Guides
SP 1200: Protocols
SP 1500: Working Group Papers
SP 1800: NIST Cybersecurity Practice Guides
SP 1900: Cyber-Physical Systems
SP 2000: Standards Coordination
SP 2100: Conference Proceedings
Technical Information on Building Materials
Technical Notes
Technology Transfer Brief

(URGENT) NIST FIPS identifiers with months parsed wrongly

Changed?	PubID	Original	Changed?	MR	DOI
TRUE	NBS FIPS PUB 107-FE	NBS FIPS 107-Feb1985	TRUE	NBS.FIPS.PUB.107-FE	NBS.FIPS.107-Feb1985
TRUE	NBS FIPS PUB 107-MAr1985	NBS FIPS 107-Mar1985	TRUE	NBS.FIPS.PUB.107-MAr1985	NBS.FIPS.107-Mar1985
TRUE	NBS FIPS PUB 108	NBS FIPS 108	TRUE	NBS.FIPS.PUB.108	NBS.FIPS.108
TRUE	NBS FIPS PUB 108-1984	NBS FIPS 108-1984	TRUE	NBS.FIPS.PUB.108-1984	NBS.FIPS.108-1984
TRUE	NBS FIPS PUB 109	NBS FIPS 109	TRUE	NBS.FIPS.PUB.109	NBS.FIPS.109
TRUE	NBS FIPS PUB 109-1985	NBS FIPS 109-1985	TRUE	NBS.FIPS.PUB.109-1985	NBS.FIPS.109-1985
TRUE	NBS FIPS PUB 110	NBS FIPS 110	TRUE	NBS.FIPS.PUB.110	NBS.FIPS.110
TRUE	NBS FIPS PUB 111	NBS FIPS 111	TRUE	NBS.FIPS.PUB.111	NBS.FIPS.111
TRUE	NBS FIPS PUB 111-1985	NBS FIPS 111-1985	TRUE	NBS.FIPS.PUB.111-1985	NBS.FIPS.111-1985
TRUE	NBS FIPS PUB 11-1eSep1977	NBS FIPS 11-1-Sep1977	TRUE	NBS.FIPS.PUB.11-1eSep1977	NBS.FIPS.11-1-Sep1977
TRUE	NBS FIPS PUB 11-1eSep30	NBS FIPS 11-1-Sep30	TRUE	NBS.FIPS.PUB.11-1eSep30	NBS.FIPS.11-1-Sep30/1977
TRUE	NBS FIPS PUB 112	NBS FIPS 112	TRUE	NBS.FIPS.PUB.112	NBS.FIPS.112
TRUE	NBS FIPS PUB 113	NBS FIPS 113	TRUE	NBS.FIPS.PUB.113	NBS.FIPS.113
TRUE	NBS FIPS PUB 114-DE	NBS FIPS 114-Dec1985	TRUE	NBS.FIPS.PUB.114-DE	NBS.FIPS.114-Dec1985
TRUE	NBS FIPS PUB 114-SE	NBS FIPS 114-Sep1985	TRUE	NBS.FIPS.PUB.114-SE	NBS.FIPS.114-Sep1985
TRUE	NBS FIPS PUB 115-MAr1985	NBS FIPS 115-Mar1985	TRUE	NBS.FIPS.PUB.115-MAr1985	NBS.FIPS.115-Mar1985
TRUE	NBS FIPS PUB 115-SE	NBS FIPS 115-Sep1985	TRUE	NBS.FIPS.PUB.115-SE	NBS.FIPS.115-Sep1985
TRUE	NBS FIPS PUB 116-A	NBS FIPS 116-Apr1985	TRUE	NBS.FIPS.PUB.116-A	NBS.FIPS.116-Apr1985
TRUE	NBS FIPS PUB 116-SE	NBS FIPS 116-Sep1985	TRUE	NBS.FIPS.PUB.116-SE	NBS.FIPS.116-Sep1985

(URGENT) Revision information missing in NIST SP 800-56 documents

PubID	Original	(ignore)	MR	DOI
NIST SP 800-56A	NIST SP 800-56ar	TRUE	NIST.SP.800-56A	NIST.SP.800-56ar
NIST SP 800-56A	NIST SP 800-56Ar2	TRUE	NIST.SP.800-56A	NIST.SP.800-56Ar2
NIST SP 800-56A	NIST SP 800-56Ar3	TRUE	NIST.SP.800-56A	NIST.SP.800-56Ar3
NIST SP 800-56B	NIST SP 800-56b	TRUE	NIST.SP.800-56B	NIST.SP.800-56b
NIST SP 800-56B	NIST SP 800-56Br1	TRUE	NIST.SP.800-56B	NIST.SP.800-56Br1
NIST SP 800-56B	NIST SP 800-56Br2	TRUE	NIST.SP.800-56B	NIST.SP.800-56Br2
NIST SP 800-56C	NIST SP 800-56c	TRUE	NIST.SP.800-56C	NIST.SP.800-56c
NIST SP 800-56C	NIST SP 800-56Cr1	TRUE	NIST.SP.800-56C	NIST.SP.800-56Cr1
NIST SP 800-56C	NIST SP 800-56Cr2	TRUE	NIST.SP.800-56C	NIST.SP.800-56Cr2

"NIST SP 800-56ar" is Revision 1 of "NIST SP 800-56A".

Extract NIST machine-readable PubID from metanorma-nist

Originally implemented as part of https://github.com/metanorma/metanorma-nist/issues/88
https://github.com/metanorma/metanorma-nist/search?q=nist-mr

The original ticket contents should become part of the README of this gem.

This gem is supposed to do the following:

Ability to parse a NIST document identifier:

NistPubid.parse("NIST SP 800-53r5") #=> NistPubid <instance>
NistPubid.new(publisher: :nist, series: "SP", docnumber: "800-53", revision: 5) #=> NistPubid <instance>

Ability to generate the NIST PubID string outputs

NistPubid.parse("NIST SP 800-53r5").to_s(:long)
NistPubid.parse("NIST SP 800-53r5").to_s(:abbrev)
NistPubid.parse("NIST SP 800-53r5").to_s(:short)

parse/convert a NIST PubID into a NIST MR PubID

NistPubid.parse("NIST SP 800-53r5").to_s(:mr)

parse/convert a NIST MR PubID into a human readable identifier

NistPubid.parse("NIST.SP.800-53.rev5").to_s(:long)

do things with the PubID object, e.g. generate DOI, parse old-style DOI

NistPubid.parse_doi("NIST.SP.800-53r5").to_s(:mr)
NistPubid.parse("NIST.SP.800-53.rev5").to_s(:doi)

access/manipulate the PubID object (e.g. get edition, revision, etc)

NistPubid.parse("NIST SP 800-53r5").publisher #=> :nist
x = NistPubid.parse("NIST SP 800-53r5")
x.revision = 6
x.revision # => 6

This gem will be used by relaton-nist and metanorma-nist.

(URGENT) Incorrect conversion for "NBS CIRC 54index"

Right now it converts as:

$ nist-pubid convert "NBS CIRC 54index"
NBS CIRC 54INDEX

But it should be NBS CIRC 54indx.

NIST data source clarification: "NBS CRPL c4-4" is incorrect (which led to parsing error)

TRUE	parse error	NBS CRPL c4-4	TRUE	parse_error	NBS.CRPL.c4-4

(URGENT) Lost `-a` information in "NIST IR 6529-a"

NIST IR 6529	NIST IR 6529-a	TRUE	NIST.IR.6529	NIST.IR.6529-a

(URGENT) `NIST.CSWP.01162020pt` is a Portuguese document but not labelled as such

NIST CSRC White Paper 01162020	NIST CSWP 01162020pt	TRUE	NIST.CSRC.White.Paper.01162020	NIST.CSWP.01162020pt	Portuguese Translation of the NIST Privacy Framework Version 1.0

NBS HB 44e2 `short_pubid` is confusing

@mico I think the short_pubid seems confusing here.

The original title is this:

This is Edition 2 of Handbook 44, published in 1955. The 1955 is the year, not edition and not part of the number.

In the 2021-08 edition of the PubID spec:

we are looking at the normal style (not the update style). There can only be one "edition".

Amongst the examples:

Either the edition number is used or the edition year is used.

So I'd say we have either:

NBS HB 44e2
NBS NB 44e1955 (if we treat 1955 as the edition number)

UPDATE: The outcome will be dependent on NIST's response to #28 .

Originally posted by @ronaldtse in #22 (comment)

NIST PubID scheme: "NBS CIRC 11e2-1915" now converted into "NBS CIRC 11-1915e2" but needs further consideration

Changed	PubID	Original	Changed	MR	DOI
TRUE	NBS CIRC 11-1915e2	NBS CIRC 11e2-1915	TRUE	NBS.CIRC.11-1915e2	NBS.CIRC.11e2-1915
TRUE	NBS CIRC 11-1917e3	NBS CIRC 11e3-1917	TRUE	NBS.CIRC.11-1917e3	NBS.CIRC.11e3-1917
TRUE	NBS CIRC 12e2	NBS CIRC 13e2revjune1908	TRUE	NBS.CIRC.12e2	NBS.CIRC.12e2revjune
TRUE	NBS CIRC 13e2	NBS CIRC 13e2revJune1908	TRUE	NBS.CIRC.13e2	NBS.CIRC.13e2revJune1908
TRUE	NBS CIRC 74-1937e1937	NBS CIRC 74e1937	TRUE	NBS.CIRC.74-1937e1937	NBS.CIRC.74-1937

(URGENT) Broken "NIST SP 800-nn{part}r{x}" when {part} exists

PubID	Original
NIST SP 800-53A	NIST SP 800-53Ar4
NIST SP 800-56A	NIST SP 800-56a
NIST SP 800-56A	NIST SP 800-56ar
NIST SP 800-56A	NIST SP 800-56Ar2
NIST SP 800-56A	NIST SP 800-56Ar3
NIST SP 800-56B	NIST SP 800-56b
NIST SP 800-56B	NIST SP 800-56Br1
NIST SP 800-56B	NIST SP 800-56Br2
NIST SP 800-56C	NIST SP 800-56c
NIST SP 800-56C	NIST SP 800-56Cr1
NIST SP 800-56C	NIST SP 800-56Cr2

i.e. "NIST SP 800-53Ar4" is correct, it should not be changed into "NIST SP 800-53A".

(URGENT) "NIST SP 800-38a-add" should be converted into "NIST SP 800-38A Add."

Now:

PubID	Original
NIST SP 800-38A Addendum	NIST SP 800-38a-add

Improvement to CLI usage and table comparison

Can we make these changes to the CLI:

In the table comparison mode, we also want to highlight which roles changed the ID:

Change header to:

Changed | New PubID | Document ID | DOI | Title + subtitle
✅ | (changed ID compared between PubID and Document ID)...
- | (no change)

Make it a real CLI using Thor and include it in gemspec as an executable under exe/nist-pubid:

$ bundle exec nist-pubid   # During development
$ nist-pubid               # After install

Instead of

$ ./nist-pubid

And I should be able to

Support CSV output
e.g.

% nist-pubid -s -f csv
# => CSV output

CLI conversion for single document ID support document ID and DOI

# Converting from document ID (-c for convert)
% nist-pubid -c "NBS BH 10"
# => NBS BH 10

# Converting from DOI (-d for DOI, -r for revision, -e for edition, --vol for version)
% nist-pubid -d "NBS.BH.10"
# => NBS BH 10

# Converting from DOI to PubID short style (-s for style)
% nist-pubid -d "NBS.BH.10" -s short
# => NBS BH 10

# Converting from DOI to all PubID styles in JSON
% nist-pubid -d "NBS.BH.10" -s all -f json
# => 
{
  "styles": {
    "short": "NBS BH 10",
    "abbrev": "",  /* not available for BH */
    "long": "National Bureau of Standards Building and Housing Report 10",
    "mr": "NBS.BH.10",
  },
  "publisher": "NBS",
  "series": "BH",
  "reportnumber": "10",
  "edition": 1,
  ...
}

Originally posted by @ronaldtse in #20 (comment)

(URGENT) Parse failure: Unable to parse "NIST Monograph 125" and "NIST Monograph 125, Supp. 1"

The same goes for when the series name is not an abbreviation.

$ bundle exec nist-pubid convert "NIST Monograph 125"
Calling `DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has been deprecated. Please call `DidYouMean.correct_error(error_name, spell_checker)' instead.
bundler: failed to load command: nist-pubid (/Users/me/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/bin/nist-pubid)
/Users/me/src/mn/nist-pubid/lib/nist_pubid/document.rb:144:in `parse': failed to parse serie for NIST Monograph 125 (NistPubid::Errors::ParseError)
	from /Users/me/src/mn/nist-pubid/exe/nist-pubid:44:in `convert'
	from /Users/me/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/gems/thor-1.0.1/lib/thor/command.rb:27:in `run'
	from /Users/me/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/gems/thor-1.0.1/lib/thor/invocation.rb:127:in `invoke_command'
	from /Users/me/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/gems/thor-1.0.1/lib/thor.rb:392:in `dispatch'
	from /Users/me/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/gems/thor-1.0.1/lib/thor/base.rb:485:in `start'
	from /Users/me/src/mn/nist-pubid/exe/nist-pubid:52:in `<top (required)>'
	from /Users/me/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/bin/nist-pubid:25:in `load'
	from /Users/me/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/bin/nist-pubid:25:in `<top (required)>'
	from /Users/me/.rbenv/versions/3.1.0/lib/ruby/3.1.0/bundler/cli/exec.rb:58:in `load'
	from /Users/me/.rbenv/versions/3.1.0/lib/ruby/3.1.0/bundler/cli/exec.rb:58:in `kernel_load'
	from /Users/me/.rbenv/versions/3.1.0/lib/ruby/3.1.0/bundler/cli/exec.rb:23:in `run'
	from /Users/me/.rbenv/versions/3.1.0/lib/ruby/3.1.0/bundler/cli.rb:484:in `exec'
	from /Users/me/.rbenv/versions/3.1.0/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
	from /Users/me/.rbenv/versions/3.1.0/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
	from /Users/me/.rbenv/versions/3.1.0/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
	from /Users/me/.rbenv/versions/3.1.0/lib/ruby/3.1.0/bundler/cli.rb:31:in `dispatch'
	from /Users/me/.rbenv/versions/3.1.0/lib/ruby/3.1.0/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
	from /Users/me/.rbenv/versions/3.1.0/lib/ruby/3.1.0/bundler/cli.rb:25:in `start'
	from /Users/me/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/gems/bundler-2.3.3/libexec/bundle:48:in `block in <top (required)>'
	from /Users/me/.rbenv/versions/3.1.0/lib/ruby/3.1.0/bundler/friendly_errors.rb:103:in `with_friendly_errors'
	from /Users/me/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/gems/bundler-2.3.3/libexec/bundle:36:in `<top (required)>'
	from /Users/me/.rbenv/versions/3.1.0/bin/bundle:25:in `load'
	from /Users/me/.rbenv/versions/3.1.0/bin/bundle:25:in `<main>'

Parsing draft stage

There are few ways how to define draft stage:

NIST SP 2PD 1800-13B
NIST SP PreD 1800-19B
NIST SP IPD 800-53r5

(from https://github.com/metanorma/nist-pubid/blob/main/README.adoc)
Where did it come from?

NIST SP(2PD) 800-188
NIST SP(IPD) 800-53r5
NIST SP(PRD) 1800-19B

(from https://www.nist.gov/system/files/documents/2021/08/26/Publication-ID-Proposal_26Aug21.pdf "3.3 PubID Examples")

SP 800-80(PD)
SP 800-57pt2r1 (2PD)
SP 800-205 (February 2019) (PD)

(in https://github.com/relaton/relaton-nist/blob/master/spec/relaton_nist_spec.rb)

I believe in the last one it should be 2PD not PD, but maybe it's also the old way how to write it, is it?

I implemented parsing for the examples from README.adoc, but now it's not working for relaton-nist examples and it's very different in the proposal. So, should we support both?

(URGENT) NBS FIPS PUB month recognition failure

PubID	Original
NBS FIPS PUB 33-MA	NBS FIPS 33-May1974
NBS FIPS PUB 33-OC	NBS FIPS 33-Oct1974
NBS FIPS PUB 82-SE	NBS FIPS 82-Sep1980
NBS FIPS PUB 85-MA	NBS FIPS 85-May1980
NBS FIPS PUB 85-NOv1980	NBS FIPS 85-Nov1980
NBS FIPS PUB 89-MA	NBS FIPS 89-May1981
NBS FIPS PUB 89-SE	NBS FIPS 89-Sep1981
NBS FIPS PUB 90-JU	NBS FIPS 90-Jul1983
NBS FIPS PUB 90-SE	NBS FIPS 90-Sep1983

Update "NIST IR" short code for series back to "NISTIR"

By looking at the documentation here https://github.com/metanorma/nist-pubid/blob/main/README.adoc you can see there are examples for NISTIR, I believe they are wrong:

NIST IR series

NIST IR 8011v3

NIST IR 8204/Upd 1:2019

NIST IR 8115(spa)

In the series list there it's "NISTIR".

First, looking at that, I thought it is mistake and updated "NISTIR" to "NIST IR" in the series.yaml
But after reading relaton-nist specs (https://github.com/relaton/relaton-nist/blob/master/spec/relaton_nist_spec.rb) I realized probably it should be "NISTIR"

Issues with running on Ubuntu 20.04

root@6db1a805a582:/packed-mn# nist-pubid 
Traceback (most recent call last):
	3: from /usr/local/bin/nist-pubid:23:in `<main>'
	2: from /usr/local/bin/nist-pubid:23:in `load'
	1: from /var/lib/gems/2.7.0/gems/nist-pubid-0.1.4/exe/nist-pubid:5:in `<top (required)>'
/var/lib/gems/2.7.0/gems/nist-pubid-0.1.4/exe/nist-pubid:5:in `require': cannot load such file -- nist_pubid (LoadError)

(URGENT) "NIST SP 801-err" should be "NIST SP 801err"

NIST SP 801-err	NIST SP 801-errata	TRUE	NIST.SP.801-err	NIST.SP.801-errata

According to the rules:

Publication date (year, month) is not properly parsed

We need to extract the publication date/year embedded in the identifiers. It's not doing that right now.

e.g.

nist-pubid convert -f json "NIST SP 304A-2017"  | jq
{
  "styles": {
    "short": "NIST SP 304A-2017",
    "abbrev": "Natl. Inst. Stand. Technol. Spec. Publ. 304A-2017",
    "long": "National Institute of Standards and Technology Special Publication 304A-2017",
    "mr": "NIST.SP.304A-2017"
  },
  "publisher": "NIST",
  "serie": "NIST SP",
  "code": "304A-2017"
}

The data model is incorrect here. It should be something like:

{
  "publisher": "NIST",
  "serie": "NIST SP",
  "number": "304"
  "part": "A"
  "year": "2017"
}

(URGENT) Parse failure: "NIST GCR 17-917-45"

| NIST | NISTGCR | Grant/Contract Reports | N/A | `NIST.GCR`
| https://nvlpubs.nist.gov/nistpubs/gcr/2017/NIST.GCR.17-917-45.pdf[NIST GCR 17-917-45]

me:~/src/mn/nist-pubid (main=): bundle exec nist-pubid convert "NIST GCR 17-917-45"
bundler: failed to load command: nist-pubid (/Users/me/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/bin/nist-pubid)
/Users/me/src/mn/nist-pubid/lib/nist_pubid/document.rb:144:in `parse': failed to parse serie for NIST GCR 17-917-45 (NistPubid::Errors::ParseError)
	from /Users/me/src/mn/nist-pubid/exe/nist-pubid:44:in `convert'
	from /Users/me/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/gems/thor-1.0.1/lib/thor/command.rb:27:in `run'

(URGENT) Incorrect conversion for "NBS CIRC 74errata"

Right now it converts as:

$ nist-pubid convert "NBS CIRC 74errata"
NBS CIRC 74ERRATA

But it should be NBS CIRC 74err.

Support Errata, Appendix, Insert in implementation

As per

metanorma / pubid-nist Goto Github PK

pubid-nist's Introduction

NIST publication identifiers ("NIST PubID")

Purpose

Historic identifier patterns

NIST PubID

Use cases to support

CLI conversion tool

Installation

Usage

Installation

Converting a legacy identifier to NIST PubID

Generating the bulk NIST Tech Pubs migration report

Elements of the PubID

Publisher

Series

Stage

Report number

Part

Edition

Translation

Update

General

Update number

Update year

PubID patterns

Presentation

Full PubID

Abbreviated PubID

Short PubID

Machine-readable PubID

Examples of PubID

pubid-nist's People

Contributors

Stargazers

Watchers

Forkers

pubid-nist's Issues

"NIST SP 250e1992app"

"NBS CIRC 74errata"

"NIST SP 801-errata"

Recommend Projects

Recommend Topics

Recommend Org