We are working on an engine to turn physical models into conceptual models (for the purpose of copying information from one notice to another). At the start of the engine’s run, we have a virtually empty conceptual model (containing nothing but 1 sole instance of ND-Root) and an XML string with values (the notice itself - e.g. one of the examples provided in the SDK). At the end of the run, the conceptual model should be a tree of nodes (or rather, node instances) and fields (field values) in such a way that it should be possible to re-generate the original XML.
For a notice of subtype X, our approach is as follows: using the visual model from X’s notice-type json file "X.json" (not to be confused with the values provided by the user collected via the UI, which we call a visual model instance, but which is not used in this part of our engine), build the conceptual model node by node by running XPath queries on the XML to select instances of nodes and values of fields.
We treat the visual model as a tree of nodes. We crawl it using a stack, seeded with the entries of "metadata" and "contents". For each node we visit, distinguish between: field, visual group or repeatable node group.
For visual groups: simply add entries of "contents" to the stack (nothing more).
For repeatable node groups and fields: run the XPath of the corresponding node/field (retrieved from fields.json) on the XML and create nodes in the conceptual model for each match. For each matched node instance, also add the entries of "contents" to the stack. (Our stack consists of instances of a structure that brings together pointers to the conceptual model, the visual model and XML elements.)
When adding a field/node to the conceptual model, it is possible that the field/node’s parent node does not yet exist. In such cases, we first try to add the parent node. The parent itself might also not exist, so we try the parent’s parent recursively until we find an ancestor that we can add. In other words: we construct a node path of "missing nodes" (= the field/node and 0 or more of its ancestors) and then add all of them to the conceptual model.
Based on the following part of the developer guide...
An input-field will never have a repeatable ancestor in the visual model unless that repeatable ancestor corresponds to a repeatable node that is also an ancestor of the corresponding field. Therefore, it is safe to assume that whenever you visit an input-field, that any repeatable ancestors will have already been visited and their corresponding nodes will have already been created in the conceptual model.
...it was our understanding that none of theses "missing nodes" should be repeatable, but while testing our engine, we have ran into situations where this does seem to be the case. For example, in SDK 1.6.0, for notice subtype 16, using cn_24_maximal as XML, our engine reports the following anomalies ("missing" yet repeatable nodes):
ND-NonEsubmission
ND-LotRestrictedDocuments
ND-LotDocumentsReference
ND-ProcedureMainClassification
ND-LotMainClassification
Indeed, "ND-ProcedureMainClassification" is repeatable but it does not appear explicitly in 16.json though "BT-262-Procedure" (which has ND-ProcedureMainClassification as parent) does.
Could this possibly be a mistake in the SDK (in the sense that "ND-ProcedureMainClassification" should be somewhere in 16.json), or are we missing something?