License: GNU Lesser General Public License v3.0
| Author: | Ladislav Lhotka <[email protected]> |
|---|---|
| Date: | 2024-08-23 |
Yangson is a Python 3 library for working with configuration and state data modelled using the YANG data modelling language.
python -m pip install yangson
Note that Yangson requires Python 3.9 or higher.
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
Given a range with at least 3 single values, e.g:
range "1|3|4";
The constraint.Intervals class contains those intervals:
[[1, 1], [3], [4, 4]]
Which seems to be unexpected in some parts of the code, e.g. when creating the node_digest:
...
yangson/datatype.py in _type_digest(self, config)
633 if self.range:
634 res["range"] = [[self.to_raw(r[0]), self.to_raw(r[1])]
--> 635 for r in self.range.intervals]
636 return res
637
yangson/datatype.py in <listcomp>(.0)
633 if self.range:
634 res["range"] = [[self.to_raw(r[0]), self.to_raw(r[1])]
--> 635 for r in self.range.intervals]
636 return res
IndexError: list index out of range
Is it possible to parse individual YANG models with the yangson library? For instance - https://github.com/YangModels/yang/blob/master/standard/ietf/RFC/ietf-ip%402014-06-16.yang
Ideally, I am looking to extract all YANG XPaths available from this YANG model, along with some attributes such as their data type, description, etc. Any tips?
As discussed at the 103 social, it would would be excellent to also support XML-encoded instance data.
My project uses supports both XML and JSON. I imagine many projects are this way...
Error received:
yangson.exceptions.RawTypeError: [/ietf-snmp:snmp/enable/enable-choice/traps/syslog] expected empty value.
YANG structure:
// snmp-server enable traps
container traps {
description
"Enable SNMP Traps";
presence true;
leaf syslog {
description
"Enable SNMP syslog traps";
type empty;
}
}
I had to change to a "boolean" type to work. The "empty" data type is defined as a boolean in YANG that is implicitly set to False unless explicitly specified otherwise
Hi,
how can I lookup values with have a minus character in their key field?
E.g.
list user {
key "user-name";
leaf user-name {
type string;
}
}
Something like value = instance_value.look_up(username=admin) does not work obviously.
Thanks!
The ArrayEntry class expects the "before" linked list to be reversed, but in InstanceNode._entry, it is created in order:
return ArrayEntry(i, LinkedList.from_list(val[:i]), ...
It should be:
return ArrayEntry(i, LinkedList.from_list(list(reversed(val[:i]))),
So if a list is accessed with a 2 or more index, going back up will reverse the start of the list.
Since commit 1c60783 doing leafreftype.from_yang("1") on an integer leafref will return None because it will use the default implementation which calls from_raw, which only accepts integers.
I guess parse_value should also be implemented.
This is related to #22. Looks like they added already an extension to flag when they use POSIX regexes:
https://github.com/openconfig/public/blob/master/release/models/openconfig-extensions.yang#L100
Should yangson honor that?
If a type has a default in a grouping, it seems that yangson does not pick it up.
See the last commit for a testcase:
https://github.com/iwanb/yangson/tree/test_grouping
If I have a statement with a multi-line description:
leaf someleaf {
type string;
description
"This is a description that just keeps on going on
and on
and on.";
}
Then yangson will wrongly preserve all the whitespace:
>>> someleaf.schema_node.description
'This is a description that just keeps on going on\n and on\n and on.'
RFC 7950 6.1.3 says:
If a double-quoted string contains a line break followed by space or
tab characters that are used to indent the text according to the
layout in the YANG file, this leading whitespace is stripped from the
string, up to and including the column of the starting double quote
character, or to the first non-whitespace character, whichever occurs
first.
So I would expect the correct interpretation of the string to be 'This is a description that just keeps on going on\nand on\nand on.'
Presumably this applies to string parsing in general, not just to descriptions. I haven't checked.
I'm trying to validate this Json data:
{ "ietf-netconf-acm:nacm": { "enable-nacm": false }, "ietf-routing:routing": { "control-plane-protocols": { "control-plane-protocol": [ { "type": "rpl", "name": "RPL", "uis-rpl:example1": 1, "uis-rpl:rpl": { "example2":250 } } ] }, "ribs": { "rib": [ ] } } }
It validates (and works) with every leaf or container outside the uis-rpl:rpl container. However, whenever i try to validate content inside the rpl container, i get this error:
File "/home/douglas/.local/lib/python3.5/site-packages/yangson/schemadata.py", line 288, in resolve_pname raise UnknownPrefix(p, mid) from None yangson.exceptions.UnknownPrefix: prefix {'example2' is not defined in ('uis-rpl', '2020-02-27')
I already tried validating, for example, the objective-function container with some data, and it throws the same error (instead of "... prefix {'example2' ..." is "... prefix {'objective-function' ...").
The following image is part of the data tree. I marked the sections which fails at validating.
I'm not sure if i'm doing something wrong or it is a bug.
I can attach the models if you need them.
When I look at the IETF model file (at https://github.com/mbj4668/pyang/blob/master/modules/ietf/ietf-netconf-acm.yang) , it appears to me that the nacm container itself is not marked as mandatory.
But Yangson seems to be parsing the container (rather than some of its leaf nodes) as mandatory. I tested this with the YANGSON CLI, but I get the same result when parsing JSON data that was translated from a NETCONF XML response using a PYANG-generated XSLT file. If the NETCONF response does not contain nacm data, then validation of the JSON data fails.
Here is an excerpt of the CLI output, with the issue highlighted in boldface:
yangson -d -p /usr/local/share/yang ./yang-library-data.json | jq -a
{
"kind": "schematree",
"mandatory": true,
"description": "Data model ID: ",
"children": {
"notifications:notification": {
"kind": "container",
"mandatory": true,
"description": "internal struct to start a notification",
"children": {
"eventTime": {
"kind": "leaf",
"mandatory": true,
"type": {
"base": "string",
"derived": "date-and-time",
"patterns": [
"\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(\.\d+)?(Z|[\+\-]\d{2}:\d{2})"
]
}
}
},
"presence": false,
"config": false
},
"ietf-netconf-acm:nacm": {
"kind": "container",
"mandatory": true,
"description": "Parameters for NETCONF access control model.",
"children": {
"enable-nacm": {
"kind": "leaf",
"description": "Enables or disables all NETCONF access control\n enforcement. If 'true', then enforcement\n is enabled. If 'false', then enforcement\n is disabled.",
"type": {
"base": "boolean"
},
"default": true
},
"read-default": {
"kind": "leaf",
"description": "Controls whether read access is granted if\n no appropriate rule is found for a\n particular read request.",
"type": {
"base": "enumeration",
"derived": "action-type",
"enums": [
"permit",
"deny"
]
},
"default": "permit"
},
"write-default": {
"kind": "leaf",
"description": "Controls whether create, update, or delete access\n is granted if no appropriate rule is found for a\n particular write request.",
"type": {
"base": "enumeration",
"derived": "action-type",
"enums": [
"permit",
"deny"
]
},
"default": "deny"
},
"exec-default": {
"kind": "leaf",
"description": "Controls whether exec access is granted if no appropriate\n rule is found for a particular protocol operation request.",
"type": {
"base": "enumeration",
"derived": "action-type",
"enums": [
"permit",
"deny"
]
},
"default": "permit"
},
"enable-external-groups": {
"kind": "leaf",
"description": "Controls whether the server uses the groups reported by the\n NETCONF transport layer when it assigns the user to a set of\n NACM groups. If this leaf has the value 'false', any group\n names reported by the transport layer are ignored by the\n server.",
"type": {
"base": "boolean"
},
"default": true
},
"denied-operations": {
"kind": "leaf",
"mandatory": true,
"description": "Number of times since the server last restarted that a\n protocol operation request was denied.",
"type": {
"base": "uint32",
"derived": "zero-based-counter32"
},
"config": false
},
"denied-data-writes": {
"kind": "leaf",
"mandatory": true,
"description": "Number of times since the server last restarted that a\n protocol operation request to alter\n a configuration datastore was denied.",
"type": {
"base": "uint32",
"derived": "zero-based-counter32"
},
"config": false
},
"denied-notifications": {
"kind": "leaf",
"mandatory": true,
"description": "Number of times since the server last restarted that\n a notification was dropped for a subscription because\n access to the event type was denied.",
"type": {
"base": "uint32",
"derived": "zero-based-counter32"
},
"config": false
},
"groups": {
"kind": "container",
"description": "NETCONF access control groups.",
"children": {
"group": {
"kind": "list",
"description": "One NACM group entry. This list will only contain\n configured entries, not any entries learned from\n any transport protocols.",
"children": {
"name": {
"kind": "leaf",
"mandatory": true,
"description": "Group name associated with this entry.",
"type": {
"base": "string",
"derived": "group-name-type",
"length": [
[
1,
4294967295
]
],
"patterns": [
"[^\\*]."
]
}
},
"user-name": {
"kind": "leaf-list",
"description": "Each entry identifies the username of\n a member of the group associated with\n this entry.",
"type": {
"base": "string",
"derived": "user-name-type",
"length": [
[
1,
4294967295
]
]
}
}
},
"keys": [
"name"
]
}
},
"presence": false
},
"rule-list": {
"kind": "list",
"description": "An ordered collection of access control rules.",
"children": {
"name": {
"kind": "leaf",
"mandatory": true,
"description": "Arbitrary name assigned to the rule-list.",
"type": {
"base": "string",
"length": [
[
1,
4294967295
]
]
}
},
"group": {
"kind": "leaf-list",
"description": "List of administrative groups that will be\n assigned the associated access rights\n defined by the 'rule' list.\n\n The string '' indicates that all groups apply to the\n entry.",
"type": {
"base": "union"
}
},
"rule": {
"kind": "list",
"description": "One access control rule.\n\n Rules are processed in user-defined order until a match is\n found. A rule matches if 'module-name', 'rule-type', and\n 'access-operations' match the request. If a rule\n matches, the 'action' leaf determines whether or not\n access is granted.",
"children": {
"name": {
"kind": "leaf",
"mandatory": true,
"description": "Arbitrary name assigned to the rule.",
"type": {
"base": "string",
"length": [
[
1,
4294967295
]
]
}
},
"module-name": {
"kind": "leaf",
"description": "Name of the module associated with this rule.\n\n This leaf matches if it has the value '' or if the\n object being accessed is defined in the module with the\n specified module name.",
"type": {
"base": "union"
},
"default": ""
},
"rpc-name": {
"kind": "leaf",
"description": "This leaf matches if it has the value '' or if\n its value equals the requested protocol operation\n name.",
"type": {
"base": "union"
}
},
"notification-name": {
"kind": "leaf",
"description": "This leaf matches if it has the value '' or if its\n value equals the requested notification name.",
"type": {
"base": "union"
}
},
"path": {
"kind": "leaf",
"mandatory": true,
"description": "Data node instance-identifier associated with the\n data node, action, or notification controlled by\n this rule.\n\n Configuration data or state data\n instance-identifiers start with a top-level\n data node. A complete instance-identifier is\n required for this type of path value.\n\n The special value '/' refers to all possible\n datastore contents.",
"type": {
"base": "string",
"derived": "node-instance-identifier"
}
},
"access-operations": {
"kind": "leaf",
"description": "Access operations associated with this rule.\n\n This leaf matches if it has the value '' or if the\n bit corresponding to the requested operation is set.",
"type": {
"base": "union"
},
"default": ""
},
"action": {
"kind": "leaf",
"mandatory": true,
"description": "The access control action associated with the\n rule. If a rule has been determined to match a\n particular request, then this object is used\n to determine whether to permit or deny the\n request.",
"type": {
"base": "enumeration",
"derived": "action-type",
"enums": [
"permit",
"deny"
]
}
},
"comment": {
"kind": "leaf",
"description": "A textual description of the access rule.",
"type": {
"base": "string"
}
}
},
"keys": [
"name"
]
}
},
"keys": [
"name"
]
}
},
"presence": false
},
...
With from_raw() can a data-instance of a YANG model be loaded. In our specific instance we load a data instance of the "ietf-interfaces" model from an IOS-XE device over RESTCONF (https:///restconf/data/ietf-interfaces:interfaces/), like the one below. This works perfectly.
{
"ietf-interfaces:interfaces": {
"interface": [
{
"name": "GigabitEthernet1",
"type": "iana-if-type:ethernetCsmacd",
"enabled": true,
"ietf-ip:ipv4": {
"address": [
{
"ip": "192.168.0.40",
"netmask": "255.255.255.0"
}
]
},
"ietf-ip:ipv6": {
}
},
{
"name": "GigabitEthernet2",
"type": "iana-if-type:ethernetCsmacd",
"enabled": false,
"ietf-ip:ipv4": {
},
"ietf-ip:ipv6": {
}
}
]
}
}
When the request is specified to only return one interface from the list (e.g. https:///restconf/data/ietf-interfaces:interfaces/interface=GigabitEthernet1), the returned JSON representation looks like the second one below:
{
"ietf-interfaces:interface": {
"name": "GigabitEthernet1",
"type": "iana-if-type:ethernetCsmacd",
"enabled": true,
"ietf-ip:ipv4": {
"address": [
{
"ip": "192.168.0.40",
"netmask": "255.255.255.0"
}
]
},
"ietf-ip:ipv6": {
}
}
}
If this instance is loaded, an exception is thrown: Illegal object member: /ietf-interfaces:interface. As far as I have seen from the documentation, no such filtered models are loaded. Is this not an intended use-case, a bug, or am I overlooking something?
I am trying to build an instance route but running into troubles because there is a YANG identifier with slashes:
pdb> self.dm.parse_resource_id("/openconfig-interfaces:interfaces/interface=xe-0/0/0")
*** yangson.exceptions.UnexpectedInput: /openconfig-interfaces:interfaces/interface=xe-0/§0/0: expected YANG identifier
I tried enclosing the identifier with escaped quotes but didn't work. Is this a bug or am I doing something wrong?
Thanks!
When I attempt to parse a set of YANG modules, I encounter the following failure on two modules which each have a comment block above the "module" statement:
File "/usr/local/lib/python3.6/dist-packages/yangson-1.3.43.dev1+g7998f39-py3.6.egg/yangson/statement.py", line 268, in statement
File "/usr/local/lib/python3.6/dist-packages/yangson-1.3.43.dev1+g7998f39-py3.6.egg/yangson/statement.py", line 254, in keyword
File "/usr/local/lib/python3.6/dist-packages/yangson-1.3.43.dev1+g7998f39-py3.6.egg/yangson/parser.py", line 235, in yang_identifier
File "/usr/local/lib/python3.6/dist-packages/yangson-1.3.43.dev1+g7998f39-py3.6.egg/yangson/parser.py", line 141, in match_regex
yangson.exceptions.UnexpectedInput: line 1, column 0: expected YANG identifier
If I move the module statement to the 1st line, then it parses properly. Pyang parses modules with comments at the top with no complaints.
Hi, I'm trying to load the ietf-arp model which needs ietf-ip model but get an error when doing so.
My library config:
{
"ietf-yang-library:modules-state": {
"module-set-id": "ae595da11ace92c0d881995fa7e56bbe86f1f48e9",
"module": [
{
"name": "ietf-arp",
"revision": "2019-02-21",
"namespace": "urn:ietf:params:xml:ns:yang:ietf-arp",
"conformance-type": "implement",
"schema": "https://raw.githubusercontent.com/YangModels/yang/master/experimental/ietf-extracted-YANG-modules/[email protected]"
},
{
"name": "ietf-ip",
"revision": "2018-02-22",
"namespace": "urn:ietf:params:xml:ns:yang:ietf-ip",
"conformance-type": "import",
"schema": "https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-ip%402018-02-22.yang"
},
{
"name": "ietf-inet-types",
"revision": "2013-07-15",
"namespace": "urn:ietf:params:xml:ns:yang:ietf-inet-types",
"conformance-type": "import",
"schema": "https://raw.githubusercontent.com/YangModels/yang/master/standard/ietf/RFC/ietf-inet-types.yang"
},
{
"name": "ietf-yang-types",
"revision": "2013-07-15",
"namespace": "urn:ietf:params:xml:ns:yang:ietf-yang-types",
"conformance-type": "import",
"schema": "https://github.com/YangModels/yang/blob/master/standard/ietf/RFC/ietf-yang-types.yang"
},
{
"name": "iana-if-type",
"revision": "2017-01-19",
"namespace": "urn:ietf:params:xml:ns:yang:iana-if-type",
"conformance-type": "implement",
"schema": "https://github.com/YangModels/yang/blob/master/experimental/odp/iana-if-type.yang"
},
{
"name": "openconfig-yang-types",
"revision": "2018-11-21",
"namespace": "urn:ietf:params:xml:ns:yang:openconfig-yang-types",
"conformance-type": "implement",
"schema": "https://github.com/openconfig/public/blob/master/release/models/types/openconfig-yang-types.yang"
},
{
"name": "openconfig-types",
"revision": "2018-11-21",
"namespace": "urn:ietf:params:xml:ns:yang:openconfig-types",
"conformance-type": "implement",
"schema": "https://github.com/openconfig/public/blob/master/release/models/types/openconfig-types.yang"
},
{
"name": "openconfig-interfaces",
"revision": "2018-11-21",
"namespace": "urn:ietf:params:xml:ns:yang:openconfig-interfaces",
"conformance-type": "implement",
"schema": "https://github.com/openconfig/public/blob/master/release/models/interfaces/openconfig-interfaces.yang"
},
{
"name": "openconfig-extensions",
"revision": "2018-10-17",
"namespace": "urn:ietf:params:xml:ns:yang:openconfig-extensions",
"conformance-type": "import",
"schema": "https://github.com/openconfig/public/blob/master/release/models/openconfig-extensions.yang"
},
{
"name": "ietf-interfaces",
"revision": "2014-05-08",
"namespace": "urn:ietf:params:xml:ns:yang:ietf-interfaces",
"conformance-type": "import",
"schema": "https://github.com/openconfig/public/blob/master/release/models/interfaces/openconfig-interfaces.yang"
},
{
"name": "openconfig-vlan",
"revision": "2018-11-21",
"namespace": "urn:ietf:params:xml:ns:yang:openconfig-vlan",
"conformance-type": "implement",
"schema": "https://github.com/openconfig/public/blob/master/release/models/vlan/openconfig-vlan.yang"
},
{
"name": "openconfig-vlan-types",
"revision": "2018-11-21",
"namespace": "urn:ietf:params:xml:ns:yang:openconfig-vlan-types",
"conformance-type": "implement",
"schema": "https://github.com/openconfig/public/blob/master/release/models/vlan/openconfig-vlan-types.yang"
},
{
"name": "openconfig-lldp",
"revision": "2018-11-21",
"namespace": "urn:ietf:params:xml:ns:yang:openconfig-lldp",
"conformance-type": "implement",
"schema": "https://github.com/openconfig/public/blob/master/release/models/lldp/openconfig-lldp.yang"
},
{
"name": "openconfig-lldp-types",
"revision": "2018-11-21",
"namespace": "urn:ietf:params:xml:ns:yang:openconfig-lldp-types",
"conformance-type": "implement",
"schema": "https://github.com/openconfig/public/blob/master/release/models/lldp/openconfig-lldp-types.yang"
}
]
}
}
What I get when running:
$ yangson ./yang/yang-library-data.json --path ./yang/yang-modules/ietf:./yang/yang-modules/openconfig
Traceback (most recent call last):
File "/usr/local/bin/yangson", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python3.7/site-packages/yangson/__main__.py", line 112, in main
dm = DataModel(yl, tuple(sp.split(":")))
File "/usr/local/lib/python3.7/site-packages/yangson/datamodel.py", line 85, in __init__
self._build_schema()
File "/usr/local/lib/python3.7/site-packages/yangson/datamodel.py", line 190, in _build_schema
self.schema._augment_stmt(aug, sctx)
File "/usr/local/lib/python3.7/site-packages/yangson/schemanode.py", line 570, in _augment_stmt
target._handle_substatements(stmt, sctx)
AttributeError: 'NoneType' object has no attribute '_handle_substatements'I'm running v1.3.39
$ yangson --version
yangson 1.3.39I did see a similar failure in #15 but that should be resolved in my current version unless I'm missing something. I'm not even sure they are related, I'm still trying to get familiar with yang modeling.
I have been trying to apply yangson to some models I have been working with, in particular Cisco IOS-XE models and OpenConfig. Openconfig models leverage many leafref, mostly referencing openconfig-interfaces and it seems that for yangson validation to succeed, the data always need to have those trees referenced in leafref.
I have been wondering if it is possible to skip this particular validation as it becomes hard to provide this additional supporting data,
Default for an instance-identifier specified in a YANG module has to be parsed as XPath. It is thus necessary to implement a special from_yang method for InstanceIdentifierType.
Error thrown: yangson.exceptions.InvalidArgument: * (Using the model [email protected]).
typedef matchall-string-type {
type string {
pattern "\*"
}
description
"The string containing a single asterisk '*' is used
to conceptually represent all possible values
for the particular leaf using this data type.";
}
Had to remove the pattern to get the output to render properly.
Is the timestamp feature of the Instance objects meant to track the timestamp referred in RFC8040?
Section 3.5.1 states:
This timestamp is only affected by configuration data resources and MUST NOT be updated for changes to non-configuration data.
(edit: removed suggested solution, as always it is a little bit more complicated)
The subtree can be selected using a resource identifier.
Hello,
A leafref with a default value generates an unhandled exception.
Example Yang file:
module test {
namespace "http://example.com/test";
prefix "t";
revision "2016-04-26";
container mycontainer {
leaf leafX {
type int32;
default 6378;
}
leaf leafref {
type leafref {
path "../leafX";
}
default 6378;
}
}
}
Exception traceback:
Traceback (most recent call last):
File "/venv3/bin/yangson", line 11, in <module>
load_entry_point('yangson', 'console_scripts', 'yangson')()
File "/venv3/lib/python3.6/site-packages/pkg_resources/__init__.py", line 565, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/venv3/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2631, in load_entry_point
return ep.load()
File "/venv3/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2291, in load
return self.resolve()
File "/venv3/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2297, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/yangson/yangson/__main__.py", line 137, in <module>
sys.exit(main())
File "/yangson/yangson/__main__.py", line 85, in main
dm = DataModel(yl, path)
File "/yangson/yangson/datamodel.py", line 85, in __init__
self._build_schema()
File "/yangson/yangson/datamodel.py", line 174, in _build_schema
self.schema_data.modules[mid].statement, sctx)
File "/yangson/yangson/schemanode.py", line 198, in _handle_substatements
method(s, sctx)
File "/yangson/yangson/schemanode.py", line 528, in _container_stmt
self._handle_child(ContainerNode(), stmt, sctx)
File "/yangson/yangson/schemanode.py", line 601, in _handle_child
super()._handle_child(node, stmt, sctx)
File "/yangson/yangson/schemanode.py", line 489, in _handle_child
node._handle_substatements(stmt, sctx)
File "/yangson/yangson/schemanode.py", line 198, in _handle_substatements
method(s, sctx)
File "/yangson/yangson/schemanode.py", line 560, in _leaf_stmt
self._handle_child(node, stmt, sctx)
File "/yangson/yangson/schemanode.py", line 489, in _handle_child
node._handle_substatements(stmt, sctx)
File "/yangson/yangson/schemanode.py", line 198, in _handle_substatements
method(s, sctx)
File "/yangson/yangson/schemanode.py", line 1132, in _default_stmt
self._default = self.type.from_yang(stmt.argument, sctx)
File "/yangson/yangson/datatype.py", line 118, in from_yang
return self.parse_value(text)
File "/yangson/yangson/datatype.py", line 109, in parse_value
return self.from_raw(text)
File "/yangson/yangson/datatype.py", line 481, in from_raw
return self.ref_type.from_raw(raw)
AttributeError: 'NoneType' object has no attribute 'from_raw'
Iwan
Given the following model:
$ cat test.json
{
"ietf-yang-library:modules-state": {
"module-set-id": "",
"module": [
{
"name": "test",
"revision": "2020-07-22",
"conformance-type": "implement"
}
]
}
}
$ cat test.yang
module test {
namespace "http://example.com";
prefix test;
revision 2020-07-22;
leaf number {
type uint64;
}
}
I would expect the following program to run successfully:
#!/usr/bin/env python
from yangson import DataModel
data_model = DataModel.from_file("./test.json")
data = {'test:number': 1}
inst = data_model.from_raw(data)
inst.validate()However:
$ ./test.py
Traceback (most recent call last):
File "./test.py", line 8, in <module>
inst = data.from_raw(yaml_input)
File "/data/developer/.virtualenvs/foo/lib/python3.6/site-packages/yangson/datamodel.py", line 109, in from_raw
cooked = self.schema.from_raw(robj)
File "/data/developer/.virtualenvs/foo/lib/python3.6/site-packages/yangson/schemanode.py", line 451, in from_raw
res[ch.iname()] = ch.from_raw(rval[qn], npath)
File "/data/developer/.virtualenvs/foo/lib/python3.6/site-packages/yangson/schemanode.py", line 850, in from_raw
raise RawTypeError(jptr, self.type.yang_type() + " value")
yangson.exceptions.RawTypeError: [/test:number] expected uint64 value
Is 1 somehow not a uint64?
All goes well if the model says uint32 rather than uint64.
Using yangson version 1.3.52.
E.g. given this library:
{
"ietf-yang-library:modules-state": {
"module-set-id": "",
"module": [
{
"name": "ietf-netconf",
"revision": "2011-06-01",
"namespace": "urn:ietf:params:xml:ns:yang:ietf-netconf",
"conformance-type": "implement"
},
{
"name": "ietf-netconf-with-defaults",
"revision": "2011-06-01",
"namespace": "urn:ietf:params:xml:ns:yang:ietf-netconf-with-defaults",
"conformance-type": "implement"
},
{
"name": "ietf-yang-types",
"revision": "2013-07-15",
"namespace": "urn:ietf:params:xml:ns:yang:ietf-yang-types",
"conformance-type": "import"
},
{
"name": "ietf-inet-types",
"revision": "2013-07-15",
"namespace": "urn:ietf:params:xml:ns:yang:ietf-inet-types",
"conformance-type": "import"
},
{
"name": "ietf-datastores",
"revision": "2018-02-14",
"namespace": "urn:ietf:params:xml:ns:yang:ietf-datastores",
"conformance-type": "import"
},
{
"name": "ietf-origin",
"revision": "2018-02-14",
"namespace": "urn:ietf:params:xml:ns:yang:ietf-origin",
"conformance-type": "implement"
},
{
"name": "ietf-yang-metadata",
"revision": "2016-08-05",
"namespace": "urn:ietf:params:xml:ns:yang:ietf-yang-metadata",
"conformance-type": "import"
},
{
"name": "ietf-netconf-nmda",
"revision": "2019-01-07",
"namespace": "urn:ietf:params:xml:ns:yang:ietf-netconf-nmda",
"conformance-type": "implement"
}
]
}
}
Where ietf-netconf-nmda augments a node in ietf-netconf behind a feature, yangson says the augment target is missing:
$ yangson -p yang/standard/ietf/RFC/ yang-library.json
Traceback (most recent call last):
File "/home/ibriquem/ws/external/yangson/.venv/bin/yangson", line 11, in <module>
load_entry_point('yangson', 'console_scripts', 'yangson')()
File "/home/ibriquem/ws/external/yangson/yangson/__main__.py", line 112, in main
dm = DataModel(yl, tuple(sp.split(":")))
File "/home/ibriquem/ws/external/yangson/yangson/datamodel.py", line 85, in __init__
self._build_schema()
File "/home/ibriquem/ws/external/yangson/yangson/datamodel.py", line 190, in _build_schema
self.schema._augment_stmt(aug, sctx)
File "/home/ibriquem/ws/external/yangson/yangson/schemanode.py", line 578, in _augment_stmt
raise MissingAugmentTarget(stmt.argument)
yangson.exceptions.MissingAugmentTarget: /nc:validate/nc:input/nc:source/nc:config-source
It says the augment target is missing, but AFAIK the augment should be applied on the schema tree without taking features into account. I think that tree is not built in yangson, so an alternative is to check at each level of the path if the node is hidden behind a feature instead of being missing, and ignore the augment in that case.
This may be related #12 but this isn't for validation, just viewing the schema tree. Currently, it seems any module in the mod path is printed when using --tree with the command line utility. Are there plans to limit this to a given module? Ideally, path lets you reference a specific model (or a new parameter for that)
The following captures an email exchange that occurred in June.
Kent writes:
When iterating over a DataModel, there are TerminalNodes of type LeafrefType. I wish to get the leafrefs original XPath expression, as it was in the YANG module (or close to it). The "path" attribute returns type LocationPath, which converts to a string as below, but is there anyway to get a "flattened" path expression?
LocationPath
LocationPath
LocationPath
LocationPath
LocationPath
LocationPath
Root
Step (child ('keystore', 'ex-module'))
Step (child ('asymmetric-keys', 'ex-module'))
Step (child ('asymmetric-key', 'ex-module'))
Step (child ('certificates', 'ex-module'))
Step (child ('certificate', 'ex-module'))
Step (child ('name', 'ex-module'))
Lada writes:
Do you need just a linear representation of the leafref path, or the exact argument of the "path" statement? In the latter case, it would be possible to store the argument with each leafref type during parsing, but then of course the namespace prefix bindings would be lost.
Kent writes:
I’m thinking that something like a data_path would be the best I could hope for. So, not exactly the original leafref path value, as the prefix is converted to the module’s name.
Lada writes:
OK, this should be possible. In fact, this may be a better choice as the output of the str method for XPath Expression..
Update: I may have meant "schema_path" (not "data_path") in my last comment above.
From what i can tell, i do not see support for deviation statement in the parsing done in SchemaData._from_yang_library()
First off, is my observation correct? Is there any intention to add this?
In practice, deviations are widely used and an important aspect in getting an accurate model of the device.
Adding the following pytest to tests/test_model.py:
def test_anydata_raw_value(data_model, instance):
ir = data_model.parse_resource_id("/test:contA/anydA")
expected = { "foo:bar": [1, 2, 3] }
rv = instance.goto(ir).raw_value()
assert(rv == expected)
And running the following command:
pytest -sxvvk test_anydata_raw_value
Produces the output:
tests/test_model.py::test_anydata_raw_value FAILED
====================================================== FAILURES =======================================================
_______________________________________________ test_anydata_raw_value ________________________________________________
data_model = <yangson.datamodel.DataModel object at 0x10e4cd220>
instance = <yangson.instance.RootNode object at 0x10e553970>
def test_anydata_raw_value(data_model, instance):
ir = data_model.parse_resource_id("/test:contA/anydA")
expected = { "foo:bar": [1, 2, 3] }
> rv = instance.goto(ir).raw_value()
tests/test_model.py:613:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
yangson/instance.py:448: in raw_value
member = self[m]
yangson/instance.py:215: in __getitem__
return self._member(key)
yangson/instance.py:585: in _member
self._member_schema_node(name), self.value.timestamp)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <yangson.instance.ObjectMember object at 0x10e553e20>, name = 'foo:bar'
def _member_schema_node(self, name: InstanceName) -> "DataNode":
qname = self.schema_node._iname2qname(name)
> res = self.schema_node.get_data_child(*qname)
E AttributeError: 'AnydataNode' object has no attribute 'get_data_child'
yangson/instance.py:614: AttributeError
=============================================== short test summary info ===============================================
FAILED tests/test_model.py::test_anydata_raw_value - AttributeError: 'AnydataNode' object has no attribute 'get_data...
We get successful validations pre 1.3.46 release and specifically before this commit 077b0b1.
Here is the relevant dm.ascii_tree():
+--rw openconfig-interfaces:interfaces
| +--rw interface* [name]
| +--rw openconfig-if-aggregate:aggregation
| | +--rw config
| | | +--rw lag-type? <aggregation-type(enumeration)>
| | | +--rw min-links? <uint16>
| | +--ro state
| | | +--ro lag-speed? <uint32>
| | | +--ro lag-type? <aggregation-type(enumeration)>
| | | +--ro member* <base-interface-ref(leafref)>
| | | +--ro min-links? <uint16>
| | +--rw openconfig-vlan:switched-vlan
| | +--rw config
| | | +--rw access-vlan? <union>
| | | +--rw interface-mode? <vlan-mode-type(enumeration)>
| | | +--rw native-vlan? <union>
| | | +--rw trunk-vlans* <union>
| | +--ro state
| | +--ro access-vlan? <union>
| | +--ro interface-mode? <vlan-mode-type(enumeration)>
| | +--ro native-vlan? <union>
| | +--ro trunk-vlans* <union>
| +--rw config
| | +--rw description? <string>
| | +--rw enabled? <boolean>
| | +--rw mtu? <uint16>
| | +--rw name? <string>
| | +--rw type <identityref>
| +--rw openconfig-if-ethernet:ethernet
| | +--rw config
| | | +--rw openconfig-if-aggregate:aggregate-id? <leafref>
| | | +--rw auto-negotiate? <boolean>
| | | +--rw duplex-mode? <enumeration>
| | | +--rw enable-flow-control? <boolean>
| | | +--rw mac-address? <mac-address(string)>
| | | +--rw port-speed? <identityref>
| | +--ro state
| | | +--ro openconfig-if-aggregate:aggregate-id? <leafref>
| | | +--ro auto-negotiate? <boolean>
| | | +--ro counters
| | | | +--ro in-8021q-frames? <counter64(uint64)>
| | | | +--ro in-crc-errors? <counter64(uint64)>
| | | | +--ro in-fragment-frames? <counter64(uint64)>
| | | | +--ro in-jabber-frames? <counter64(uint64)>
| | | | +--ro in-mac-control-frames? <counter64(uint64)>
| | | | +--ro in-mac-pause-frames? <counter64(uint64)>
| | | | +--ro in-oversize-frames? <counter64(uint64)>
| | | | +--ro out-8021q-frames? <counter64(uint64)>
| | | | +--ro out-mac-control-frames? <counter64(uint64)>
| | | | +--ro out-mac-pause-frames? <counter64(uint64)>
| | | +--ro duplex-mode? <enumeration>
| | | +--ro effective-speed? <uint32>
| | | +--ro enable-flow-control? <boolean>
| | | +--ro hw-mac-address? <mac-address(string)>
| | | +--ro mac-address? <mac-address(string)>
| | | +--ro negotiated-duplex-mode? <enumeration>
| | | +--ro negotiated-port-speed? <identityref>
| | | +--ro port-speed? <identityref>
| | +--rw openconfig-vlan:switched-vlan
| | +--rw config
| | | +--rw access-vlan? <union>
| | | +--rw interface-mode? <vlan-mode-type(enumeration)>
| | | +--rw native-vlan? <union>
| | | +--rw trunk-vlans* <union>
| | +--ro state
| | +--ro access-vlan? <union>
| | +--ro interface-mode? <vlan-mode-type(enumeration)>
| | +--ro native-vlan? <union>
| | +--ro trunk-vlans* <union>
Here is our data we loaded:
{
"openconfig-interfaces:interfaces": {
"interface": [
{
"name": "FastEthernet1",
"config": {
"type": "iana-if-type:ethernetCsmacd",
"name": "FastEthernet1",
"description": "This is Fa1",
"enabled": false
},
"subinterfaces": {
"subinterface": [
{
"index": 1,
"config": {
"index": 1,
"description": "This is Fa1.1"
}
},
{
"index": 2,
"config": {
"index": 2,
"description": "This is Fa1.2"
}
}
]
}
},
{
"name": "FastEthernet3",
"config": {
"type": "iana-if-type:ethernetCsmacd",
"name": "FastEthernet3",
"description": "This is Fa3",
"enabled": true
},
"openconfig-if-ethernet:ethernet": {
"openconfig-vlan:switched-vlan": {
"config": {
"interface-mode": "ACCESS",
"access-vlan": 10
}
}
}
},
{
"name": "FastEthernet4",
"config": {
"type": "iana-if-type:ethernetCsmacd",
"name": "FastEthernet4",
"enabled": false
},
"openconfig-if-ethernet:ethernet": {
"openconfig-vlan:switched-vlan": {
"config": {
"interface-mode": "TRUNK",
"trunk-vlans": [
10,
20
]
}
}
}
}
]
},
"openconfig-vlan:vlans": {
"vlan": [
{
"vlan-id": 10,
"config": {
"vlan-id": 10,
"name": "prod",
"status": "ACTIVE"
}
},
{
"vlan-id": 20,
"config": {
"vlan-id": 20,
"name": "dev",
"status": "SUSPENDED"
}
}
]
}
}
The error we get is: *** yangson.exceptions.SchemaError: [/openconfig-interfaces:interfaces/interface/1] config member-not-allowed: openconfig-if-ethernet:ethernet
>>> irt = self.dm.parse_resource_id('/openconfig-interfaces:interfaces/interface=FastEthernet3')
>> obj.value
{'openconfig-interfaces:interfaces': {'interface': [{'name': 'FastEthernet1', 'config': {'type': ('ethernetCsmacd', 'iana-if-type'), 'name': 'FastEthernet1', 'description': 'This is Fa1', 'enabled': False}, 'subinterfaces': {'subinterface': [{'index': 1, 'config': {'index': 1, 'description': 'This is Fa1.1'}}, {'index': 2, 'config': {'index': 2, 'description': 'This is Fa1.2'}}]}}, {'name': 'FastEthernet3', 'config': {'type': ('ethernetCsmacd', 'iana-if-type'), 'name': 'FastEthernet3', 'description': 'This is Fa3', 'enabled': True}, 'openconfig-if-ethernet:ethernet': {'openconfig-vlan:switched-vlan': {'config': {'interface-mode': 'ACCESS', 'access-vlan': 10}}}}, {'name': 'FastEthernet4', 'config': {'type': ('ethernetCsmacd', 'iana-if-type'), 'name': 'FastEthernet4', 'enabled': False}, 'openconfig-if-ethernet:ethernet': {'openconfig-vlan:switched-vlan': {'config': {'interface-mode': 'TRUNK', 'trunk-vlans': [10, 20]}}}}]}, 'openconfig-vlan:vlans': {'vlan': [{'vlan-id': 10, 'config': {'vlan-id': 10, 'name': 'prod', 'status': 'ACTIVE'}}, {'vlan-id': 20, 'config': {'vlan-id': 20, 'name': 'dev', 'status': 'SUSPENDED'}}]}}
>>> fa3 = obj.goto(irt)
>>> fa3.path
('openconfig-interfaces:interfaces', 'interface', 1)
>>> fa3.json_pointer()
'/openconfig-interfaces:interfaces/interface/1'
>>> fa3.value
{'name': 'FastEthernet3', 'config': {'type': ('ethernetCsmacd', 'iana-if-type'), 'name': 'FastEthernet3', 'description': 'This is Fa3', 'enabled': True}, 'openconfig-if-ethernet:ethernet': {'openconfig-vlan:switched-vlan': {'config': {'interface-mode': 'ACCESS', 'access-vlan': 10}}}}I'm trying to provide as much information to hopefully help fix the issue. I'm not sure if we need to do something different on our end or not.
Thanks in advance for the help.
I'm trying to work with a model containing a 'must' statement which checks the equality of a leaf within a list entry.
The example-statement looks like:
must "/mod:container1/mod:list-a[mod:name=.]/mod:identifier=../identifier"
Example yang-file:
https://github.com/michael-stemmler/yangson-test/blob/master/yangmodule/must-statement/example-module.yang#L38
Using this model with this example-data gives an SemanticError exception
def _check_must(self, inst: "InstanceNode") -> None:
for m in self.must:
if not m.expression.evaluate(inst):
raise SemanticError(inst.json_pointer(), m.error_tag,
> m.error_message)
E yangson.exceptions.SemanticError: [/example-module:container2/list-b/0/list-a-name] must-violation
../../../../venv/testyangson/lib/python3.6/site-packages/yangson/schemanode.py:783: SemanticError
When changing the statement to
must "/mod:container1/mod:list-a[.]/mod:identifier=../identifier"
my test runs successful.
But I think the first case is also a valid expression.
Thanks
Hello,
I am trying to load some Openconfig models and I am having problems with some augmentations that have when statements. For instance:
The error:
self = <yangson.schemanode.SchemaTreeNode object at 0x7f1df2c17898>, stmt = <yangson.statement.Statement object at 0x7f1df2ad2160>, sctx = <yangson.schemadata.SchemaContext object at 0x7f1df2a5c5f8>
def _augment_stmt(self, stmt: Statement, sctx: SchemaContext) -> None:
"""Handle **augment** statement."""
if not sctx.schema_data.if_features(stmt, sctx.text_mid):
return
path = sctx.schema_data.sni2route(stmt.argument, sctx)
target = self.get_schema_descendant(path)
if stmt.find1("when"):
gr = GroupNode()
target._add_child(gr)
target = gr
> target._handle_substatements(stmt, sctx)
E AttributeError: 'NoneType' object has no attribute '_handle_substatements'
Removing the when will make it work. Is this a problem with the model or with yangson?
Regards
Enable Yangson to validate instance documents that have been modeled using the sx:structure extension statement. Likely SHOULD also support the sx:augment-structure extension.
Bonus: also support RFC 8040's rc:yang-data extension statement. This may seem no longer important, keep in mind that the RESTCONF protocol is very important and accessing its yang-errors and yang-api models is something I do often, albeit after hacking the YANG module to expose the inner-container as a top-level config false node...
Hi
I'm trying to build json library yang file using mkylib tool, but it I'm unable to get it work. I'm sure I'm doing something wrong:
$ python yangson/tools/python/mkylib.py yang-models/standard/ietf/RFC
Traceback (most recent call last):
File "yangson/tools/python/mkylib.py", line 112, in <module>
sys.exit(main())
File "yangson/tools/python/mkylib.py", line 71, in main
module_entry(yf)
File "yangson/tools/python/mkylib.py", line 35, in module_entry
mst = mp.statement()
File "/home/ubuntu/miniconda3/envs/python37/lib/python3.7/site-packages/yangson/statement.py", line 286, in statement
res.substatements = self.substatements()
File "/home/ubuntu/miniconda3/envs/python37/lib/python3.7/site-packages/yangson/statement.py", line 390, in substatements
res.append(self.statement())
File "/home/ubuntu/miniconda3/envs/python37/lib/python3.7/site-packages/yangson/statement.py", line 286, in statement
res.substatements = self.substatements()
File "/home/ubuntu/miniconda3/envs/python37/lib/python3.7/site-packages/yangson/statement.py", line 390, in substatements
res.append(self.statement())
File "/home/ubuntu/miniconda3/envs/python37/lib/python3.7/site-packages/yangson/statement.py", line 286, in statement
res.substatements = self.substatements()
File "/home/ubuntu/miniconda3/envs/python37/lib/python3.7/site-packages/yangson/statement.py", line 390, in substatements
res.append(self.statement())
File "/home/ubuntu/miniconda3/envs/python37/lib/python3.7/site-packages/yangson/statement.py", line 281, in statement
sub = self.argument()
File "/home/ubuntu/miniconda3/envs/python37/lib/python3.7/site-packages/yangson/statement.py", line 302, in argument
self.dq_argument()
File "/home/ubuntu/miniconda3/envs/python37/lib/python3.7/site-packages/yangson/statement.py", line 353, in dq_argument
if self._escape else self.input[start:self.offset])
File "/home/ubuntu/miniconda3/envs/python37/lib/python3.7/site-packages/yangson/statement.py", line 194, in unescape
raise InvalidArgument(text) from None
yangson.exceptions.InvalidArgument: \S+
I'm trying to parse directory https://github.com/YangModels/yang/tree/master/standard/ietf/RFC
Thanks,
J.
Using 1.3.45 I am able to validate a YAML file, but using 1.3.47 YANGSON (released today) it errors:
Validating sample/sdf-rpas.yaml..
Traceback (most recent call last):
File "build/validate.py", line 17, in
instance.validate()
File "/data/developer/.virtualenvs/AUT-949/lib/python3.6/site-packages/yangson/instance.py", line 336, in validate
self.schema_node._validate(self, scope, ctype)
File "/data/developer/.virtualenvs/AUT-949/lib/python3.6/site-packages/yangson/schemanode.py", line 485, in _validate
inst._member(m).validate(scope, ctype)
File "/data/developer/.virtualenvs/AUT-949/lib/python3.6/site-packages/yangson/instance.py", line 336, in validate
self.schema_node._validate(self, scope, ctype)
File "/data/developer/.virtualenvs/AUT-949/lib/python3.6/site-packages/yangson/schemanode.py", line 801, in _validate
super()._validate(inst, scope, ctype)
File "/data/developer/.virtualenvs/AUT-949/lib/python3.6/site-packages/yangson/schemanode.py", line 485, in _validate
inst._member(m).validate(scope, ctype)
File "/data/developer/.virtualenvs/AUT-949/lib/python3.6/site-packages/yangson/instance.py", line 336, in validate
self.schema_node._validate(self, scope, ctype)
File "/data/developer/.virtualenvs/AUT-949/lib/python3.6/site-packages/yangson/schemanode.py", line 977, in _validate
super()._validate(e, scope, ctype)
File "/data/developer/.virtualenvs/AUT-949/lib/python3.6/site-packages/yangson/schemanode.py", line 801, in _validate
super()._validate(inst, scope, ctype)
File "/data/developer/.virtualenvs/AUT-949/lib/python3.6/site-packages/yangson/schemanode.py", line 485, in _validate
inst._member(m).validate(scope, ctype)
File "/data/developer/.virtualenvs/AUT-949/lib/python3.6/site-packages/yangson/instance.py", line 336, in validate
self.schema_node._validate(self, scope, ctype)
File "/data/developer/.virtualenvs/AUT-949/lib/python3.6/site-packages/yangson/schemanode.py", line 977, in _validate
super()._validate(e, scope, ctype)
File "/data/developer/.virtualenvs/AUT-949/lib/python3.6/site-packages/yangson/schemanode.py", line 801, in _validate
super()._validate(inst, scope, ctype)
File "/data/developer/.virtualenvs/AUT-949/lib/python3.6/site-packages/yangson/schemanode.py", line 485, in _validate
inst._member(m).validate(scope, ctype)
File "/data/developer/.virtualenvs/AUT-949/lib/python3.6/site-packages/yangson/instance.py", line 336, in validate
self.schema_node._validate(self, scope, ctype)
File "/data/developer/.virtualenvs/AUT-949/lib/python3.6/site-packages/yangson/schemanode.py", line 801, in _validate
super()._validate(inst, scope, ctype)
File "/data/developer/.virtualenvs/AUT-949/lib/python3.6/site-packages/yangson/schemanode.py", line 483, in _validate
self._check_schema_pattern(inst, ctype)
File "/data/developer/.virtualenvs/AUT-949/lib/python3.6/site-packages/yangson/schemanode.py", line 506, in _check_schema_pattern
"member-not-allowed", m)
yangson.exceptions.SchemaError: [/msw-deployment:deployment/sites/0/vnfcs/0/vnfc-vsphere-options] config member-not-allowed: datastore
where vnfc-vsphere-options has the following container:
container vnfc-vsphere-options {
when "ancestor::sites/site-parameters/vim-configuration/type = 'vsphere'";
description
"vSphere-specific options for this VNFC";
uses vsphere:vsphere-options-per-vnfc;
uses vsphere:vsphere-options-only-per-vnfci {
when "derived-from(../../type, 'util:standalone-product')";
}
}
and datastore is define only in vsphere:vsphere-options-only-per-vnfci
Assuming a data model like:
+--rw boot-images
+--rw boot-image* [name]
+--rw name string
+--rw os-name? string
+--rw os-version? string
+--rw download-uri* ietf-inet-types:uri
+--rw image-verification* [hash-algorithm]
+--rw hash-algorithm identityref
+--rw hash-value ietf-yang-types:hex-string
And code like:
print("data_path = " + data_path)
irt = self.dm.parse_resource_id(data_path)
print("irt = " + str(irt))
try:
in_node = self.inst.goto(irt)
except yangson.exceptions.NonexistentInstance as e:
print("Data node (" + data_path + ") does not exist. " + str(e))
inst_obj = self.inst.raw_value()
print("inst_objj['foo:boot-images'] = " + json.dumps(inst_obj["foo:boot-images"], indent=4))
A particular run generates this output (additional line-returns added for clarity):
data_path = /foo:boot-images/boot-image=vendoros-19.2r1b6.img/download-uri=https%3A%2F%2Fcdn3.example.com%2Fpath%2Fto%2Fimage%2Ffile
irt = /foo:boot-images/boot-image[name="vendoros-19.2r1b6.img"]/download-uri[.="https://cdn3.example.com/path/to/image/file"]
Data node (/foo:boot-images/boot-image=vendoros-19.2r1b6.img/download-uri=https%3A%2F%2Fcdn3.example.com%2Fpath%2Fto%2Fimage
%2Ffile) does not exist. [/foo:boot-images/boot-image/0/download-uri] entry 'https://cdn3.example.com/path/to/image/file'
inst_obj['foo:boot-images'] = {
"boot-image": [
{
"name": "vendoros-19.2r1b6.img",
"os-name": "VendorOS",
"os-version": "19.2r1b6",
"image-verification": [
{
"hash-algorithm": "ietf-sztp-conveyed-info:sha-256",
"hash-value": "ba:ec:cf:a5:67:82:b4:10:77:c6:67:a6:22:ab:7d:50:04:a7:8b:8f:0e:db:02:8b:f4:75:55:fb:c1:13:b2:33"
}
],
"download-uri": [
"https%3A%2F%2Fcdn1.example.com%2Fpath%2Fto%2Fimage%2Ffile",
"https%3A%2F%2Fcdn2.example.com%2Fpath%2Fto%2Fimage%2Ffile",
"https%3A%2F%2Fcdn3.example.com%2Fpath%2Fto%2Fimage%2Ffile",
"https%3A%2F%2Fcdn4.example.com%2Fpath%2Fto%2Fimage%2Ffile",
"https%3A%2F%2Fcdn5.example.com%2Fpath%2Fto%2Fimage%2Ffile"
]
}
]
}
As can be seen, https%3A%2F%2Fcdn3.example.com%2Fpath%2Fto%2Fimage%2Ffile exists!
Is the URL percent-encoding causing the problem? Note that the "irt" output renders the decoded string...so maybe it's looking up the wrong value?
Precondition:
yangson multiple times (distinct invocations)Postcondition:
yangson that would just output the report (e.g., tree diagram)Hi... I just discovered a few 'strange' things while working on xml/json import/export and my own Restconf server and the RPC implementation. My main issue at the moment is that InputNode and OutputNode are not DataNodes... but the RPC input and output contains the "input/output" tag. While you can get around this with stripping of one layer of JSON, doing the from/to-json/xml part and then re-adding them, making them inhert DataNode seems to be more clean...
but this lead me to a single yangson testcase that goes wrong... the ascii-tree testcase.
at the end you have the 'correct' output:
+---x testb:rpcA
+--ro input
| +--ro testb:leafK? <typA(int16)>
+--ro output
+--ro testb:llistC* <boolean>
is this correct? the rpcA node has the same namespace than the childs of input/output, but you have the explicit namespace.
I noticed this namespace vanishes when you change InputNode/Output node to inhert from DataNode.
What do you think?
Suppose I have a DataModel, and an instance (created via from_raw()).
As I traverse the instance, is there a way for me to find the Statement corresponding to a SchemaNode?
This may be an XY question, so here's my use case - perhaps I'm missing some altogether better way.
I want to define an extension:
extension sensitive {
description
"A new YANG keyword used to indicate that a node should be considered sensitive.";
}and then in my model I want to mark some nodes as sensitive eg
leaf secret {
type string;
my-prefix:sensitive;
}And now as I traverse my instance, I want to be able to tell whether I have a sensitive node or not.
Experiment shows that yangson is happy to treat my extension as valid and turn it into a Statement, but I'm struggling to access that information when I need it.
What I have made work is monkey-patching the SchemaNode, something like this:
def my_sensitive_stmt(self, stmt, sctx):
self.sensitive = True
SchemaNode._my_sensitive_stmt = my_sensitive_stmt
SchemaNode._stmt_callback["my-prefix:sensitive"] = "_my_sensitive_stmt"Now the node gets a sensitive attribute, if it is marked sensitive in the schema.
Am I missing an easier way to do this? If not: would yangson have any interest in exposing an API that allowed me to avoid this monkey-patching?
Adding the following pytest to tests/test_model.py:
def test_empty_raw_value(data_model, instance):
ir = data_model.parse_resource_id("/test:contA/listA=C0FFEE,true/contD/contE")
expected = {'leafJ': [None], 'leafP': 10}
rv = instance.goto(ir).raw_value()
assert(rv == expected)
And running the following command:
pytest -sxvvk test_empty_raw_value
Produces the output:
> assert(rv == expected)
E AssertionError: assert {'leafJ': (None,), 'leafP': 10} == {'leafJ': [None], 'leafP': 10}
E Common items:
E {'leafP': 10}
E Differing items:
E {'leafJ': (None,)} != {'leafJ': [None]}
E Full diff:
E - {'leafJ': [None], 'leafP': 10}
E ? ^ -
E + {'leafJ': (None,), 'leafP': 10}
E ? ^ ++
I keep trying to make yangson validate openconfig models and when using models with include statements things seem to break. For instance:
https://github.com/openconfig/public/blob/master/release/models/bgp/openconfig-bgp.yang#L16
The error I get is:
yangson.exceptions.StatementNotFound: `prefix' in `submodule "openconfig-bgp-global" { ... }'
If I add the prefix statement then the model that includes the file raise the following error:
yangson.exceptions.DefinitionNotFound: grouping bgp-global-base
such grouping was defined in the file included.
Thanks
I was trying to start jetconf server and got the following error while populating yang modules dir:
(jetconf_test) [admin@agromov jetconf_test]$ ~/anaconda3/envs/jetconf_test/bin/jetconf -c config.yml
2018-04-03 19:11:53,901 INFO Jetconf version 0.3.4
2018-04-03 19:11:53,903 INFO Using config:
GLOBAL:
BACKEND_PACKAGE: jetconf_jukebox
DATA_JSON_FILE: /home/admin/Dropbox/Dev/Python/jetconf_test/data.json.jb
LOGFILE: '-'
LOG_DBG_MODULES:
- usr_conf_data_handlers
- knot_api
- nacm
- data
LOG_LEVEL: debug
PERSISTENT_CHANGES: false
PIDFILE: /home/admin/Dropbox/Dev/Python/jetconf_test/jetconf.pid
TIMEZONE: Europe/Moscow
VALIDATE_TRANSACTIONS: true
YANG_LIB_DIR: /home/admin/Dropbox/Dev/Python/jetconf_test/yang
HTTP_SERVER:
API_ROOT: /restconf
API_ROOT_RUNNING: /restconf_running
CA_CERT: /home/admin/Dropbox/Dev/Python/jetconf_test/ca.pem
DBG_DISABLE_CERTS: false
DOC_DEFAULT_NAME: index.html
DOC_ROOT: /home/admin/Dropbox/Dev/Python/jetconf_test/www
LISTEN_LOCALHOST_ONLY: false
PORT: 8443
SERVER_NAME: jetconf-h2
SERVER_SSL_CERT: /home/admin/Dropbox/Dev/Python/jetconf_test/server.crt
SERVER_SSL_PRIVKEY: /home/admin/Dropbox/Dev/Python/jetconf_test/server.key
UPLOAD_SIZE_LIMIT: 1
NACM:
ALLOWED_USERS:
- [email protected]
ENABLED: true
Traceback (most recent call last):
File "/home/admin/anaconda3/envs/jetconf_test/bin/jetconf", line 11, in <module>
sys.exit(main())
File "/home/admin/anaconda3/envs/jetconf_test/lib/python3.6/site-packages/jetconf/__main__.py", line 144, in main
jc.init()
File "/home/admin/anaconda3/envs/jetconf_test/lib/python3.6/site-packages/jetconf/jetconf.py", line 62, in init
datamodel = DataModel(yang_lib_str, [yang_mod_dir])
File "/home/admin/anaconda3/envs/jetconf_test/lib/python3.6/site-packages/yangson/datamodel.py", line 84, in __init__
self.schema_data = SchemaData(self.yang_library, mod_path)
File "/home/admin/anaconda3/envs/jetconf_test/lib/python3.6/site-packages/yangson/schemadata.py", line 94, in __init__
self._from_yang_library(yang_lib)
File "/home/admin/anaconda3/envs/jetconf_test/lib/python3.6/site-packages/yangson/schemadata.py", line 122, in _from_yang_library
mod = self._load_module(name, rev)
File "/home/admin/anaconda3/envs/jetconf_test/lib/python3.6/site-packages/yangson/schemadata.py", line 154, in _load_module
res = ModuleParser(infile.read()).parse()
File "/home/admin/anaconda3/envs/jetconf_test/lib/python3.6/site-packages/yangson/statement.py", line 144, in parse
res = self.statement()
File "/home/admin/anaconda3/envs/jetconf_test/lib/python3.6/site-packages/yangson/statement.py", line 262, in statement
res.substatements = self.substatements()
File "/home/admin/anaconda3/envs/jetconf_test/lib/python3.6/site-packages/yangson/statement.py", line 366, in substatements
res.append(self.statement())
File "/home/admin/anaconda3/envs/jetconf_test/lib/python3.6/site-packages/yangson/statement.py", line 262, in statement
res.substatements = self.substatements()
File "/home/admin/anaconda3/envs/jetconf_test/lib/python3.6/site-packages/yangson/statement.py", line 366, in substatements
res.append(self.statement())
File "/home/admin/anaconda3/envs/jetconf_test/lib/python3.6/site-packages/yangson/statement.py", line 262, in statement
res.substatements = self.substatements()
File "/home/admin/anaconda3/envs/jetconf_test/lib/python3.6/site-packages/yangson/statement.py", line 366, in substatements
res.append(self.statement())
File "/home/admin/anaconda3/envs/jetconf_test/lib/python3.6/site-packages/yangson/statement.py", line 257, in statement
sub = self.argument()
File "/home/admin/anaconda3/envs/jetconf_test/lib/python3.6/site-packages/yangson/statement.py", line 278, in argument
self.dq_argument()
File "/home/admin/anaconda3/envs/jetconf_test/lib/python3.6/site-packages/yangson/statement.py", line 329, in dq_argument
if self._escape else self.input[start:self.offset])
File "/home/admin/anaconda3/envs/jetconf_test/lib/python3.6/site-packages/yangson/statement.py", line 171, in unescape
raise InvalidArgument(text) from None
yangson.exceptions.InvalidArgument: \*
I use yang data models from here: https://github.com/YangModels/yang
The only difference to make it work is replacing " with ' :
(jetconf_test) [admin@agromov jetconf_test]$ diff yang_models_repo/standard/ietf/RFC/[email protected] yang/[email protected]
103c103
< pattern "\*";
---
> pattern '\*';
144c144
< pattern "[^\*].*";
---
> pattern '[^\*].*';
I think that standard yang modules should be used without any modifications.
If a YANG module has:
feature foo;
feature bar {
if-feature 'foo';
}
and the yang-library contains: (note: dependent feature foo not enabled)
"feature": [
"bar",
]
Then a stacktrace like this is shown:
...
dm=yangson.DataModel(json.dumps(yl_obj), [path])
File "/Users/kent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/yangson/datamodel.py", line 84, in __init__
self.schema_data = SchemaData(self.yang_library, mod_path
File "/Users/kent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/yangson/schemadata.py", line 101, in __init__
self._from_yang_library(yang_lib)
File "/Users/kent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/yangson/schemadata.py", line 150, in _from_yang_library
self._check_feature_dependences()
File "/Users/kent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/yangson/schemadata.py", line 216, in _check_feature_dependences
raise FeaturePrerequisiteError(*fn)
TypeError: __init__() takes 3 positional arguments but 27 were given
The logic for converting raw to uint64 seems wrong. The comment makes sense, but the implementation differs.
A raw instance may be either string or integer.
"""
if not isinstance(raw, str) or isinstance(raw, bool):
return NoneFor actual integers, None is returned because it is not a str. The check for bool does not make sense to me, and also would never be evaluated.
Suggest change to
A raw instance may be either string or integer.
"""
if not isinstance(raw, str) and not isinstance(raw, int):Hi,
I am trying to use Yangson in Sonic and as part of my experiment, I am trying to load huge instance data for validation. Although validation goes through but it is taking huge time to complete.
In my case I have sample yang for ACL. I am trying to create 1000 ACLs with 5 rules each and yang has one must expression.
Is there any way to improve performance. Can you please look into this case how to make yangson highly performant. Practically instance data will be huge in the production system.
Attaching a tar file containing dependent yang models along with instance data JSON file (acl_1000.json) along with metadata file (yang-library-ext2.json).
Command line:
bash-3.2$ time yangson yang-library-ext2.json -p ../common/:. -v acl_1000.json
real 62m59.614s
user 62m45.841s
sys 0m4.968s
bash-3.2$ yangson -V
yangson 1.3.43
An RPC statement with nacm:default-deny-all fails to get _nacm_default_deny_stmt.
Can you please check it.
Excerpt from Yang:
rpc delete-config {
nacm:default-deny-all;
description
"Delete a configuration datastore.";
reference "RFC 6241, Section 7.4";
input {
container target {
description
"Particular configuration to delete.";
choice config-target {
mandatory true;
description
"The configuration target to delete.";
leaf startup {
if-feature startup;
type empty;
description
"The startup configuration is the config target.";
}
leaf url {
if-feature url;
type inet:uri;
description
"The URL-based configuration is the config target.";
}
}
}
}
}
ipdb> mname
'_nacm_default_deny_stmt'
ipdb> getattr(self, mname)
*** AttributeError: 'RpcActionNode' object has no attribute '_nacm_default_deny_stmt'
To fix this i had to add the function _nacm_default_deny_stmt to the RPCActionNode:
class RpcActionNode(SchemaTreeNode):
"""RPC or action node."""
def __init__(self):
"""Initialize the class instance."""
super().__init__()
self._ctype = ContentType.nonconfig
self.default_deny = DefaultDeny.none # type: "DefaultDeny"
def _handle_substatements(self, stmt: Statement, sctx: SchemaContext) -> None:
self._add_child(InputNode(sctx.default_ns))
self._add_child(OutputNode(sctx.default_ns))
super()._handle_substatements(stmt, sctx)
def _flatten(self) -> List[SchemaNode]:
return [self]
def _tree_line_prefix(self) -> str:
return super()._tree_line_prefix() + "-x"
def _input_stmt(self, stmt: Statement, sctx: SchemaContext) -> None:
"""Handle RPC or action input statement."""
self.get_child("input")._handle_substatements(stmt, sctx)
def _output_stmt(self, stmt: Statement, sctx: SchemaContext) -> None:
"""Handle RPC or action output statement."""
self.get_child("output")._handle_substatements(stmt, sctx)
def _nacm_default_deny_stmt(self, stmt: Statement, sctx: SchemaContext) -> None:
"""Set NACM default access."""
if stmt.keyword == "default-deny-all":
self.default_deny = DefaultDeny.all
elif stmt.keyword == "default-deny-write":
self.default_deny = DefaultDeny.write
Yangson considers the Boolean true and false as valid values for integer type nodes.
For example, with this minimal schema:
$ cat test.json
{
"ietf-yang-library:modules-state": {
"module-set-id": "",
"module": [
{
"name": "test",
"revision": "1.0.0",
"conformance-type": "implement"
}
]
}
}
$ cat test.yang
module test {
prefix ;
revision 1.0.0;
description "Test schema.";
leaf number {
type int32;
}
}
Then I'd expect the following code to raise an exception, but it parses happily:
from yangson import DataModel
import yaml
data = DataModel.from_file("./test.json")
yaml_input = yaml.safe_load("'test:number': true")
inst = data.from_raw(yaml_input)
inst.validate()Replacing true with any string or non-integer value triggers a RawTypeError as expected.
Using yangson 1.3.43.
This should never happen:
File <my code...>
remote_node = self.dm.get_schema_node(location_schema_path)
File "/Users/kent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/yangson/datamodel.py", line 125, in get_schema_node
self.schema_data.path2route(path))
File "/Users/kent/.pyenv/versions/3.7.3/lib/python3.7/site-packages/yangson/schemanode.py", line 383, in get_schema_descendant
node = node.get_child(*p)
AttributeError: 'LeafNode' object has no attribute 'get_child'
Hello,
I wanted to try the library with some custom Yang files, where a grouping is defined in a submodule but used in the main module. E.g., the main module:
module test {
yang-version "1.1";
namespace "http://example.com/test";
prefix "t";
include subtest;
uses mygrouping;
}
The submodule:
submodule subtest {
yang-version "1.1";
belongs-to test {
prefix "t";
}
grouping mygrouping {
leaf leafX {
type int32;
default 6378;
}
}
}
But the grouping definition is not found:
Traceback (most recent call last):
File "/bin/yangson", line 11, in <module>
load_entry_point('yangson==1.3.17', 'console_scripts', 'yangson')()
File "/lib/python3.6/site-packages/pkg_resources/__init__.py", line 565, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2631, in load_entry_point
return ep.load()
File "/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2291, in load
return self.resolve()
File "/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2297, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/lib/python3.6/site-packages/yangson/__main__.py", line 137, in <module>
sys.exit(main())
File "/lib/python3.6/site-packages/yangson/__main__.py", line 85, in main
dm = DataModel(yl, path)
File "/lib/python3.6/site-packages/yangson/datamodel.py", line 85, in __init__
self._build_schema()
File "/lib/python3.6/site-packages/yangson/datamodel.py", line 174, in _build_schema
self.schema_data.modules[mid].statement, sctx)
File "/lib/python3.6/site-packages/yangson/schemanode.py", line 198, in _handle_substatements
method(s, sctx)
File "/lib/python3.6/site-packages/yangson/schemanode.py", line 514, in _uses_stmt
grp, gid = sctx.schema_data.get_definition(stmt, sctx)
File "/lib/python3.6/site-packages/yangson/schemadata.py", line 405, in get_definition
return (stmt.get_definition(loc, kw), sctx)
File "/lib/python3.6/site-packages/yangson/statement.py", line 117, in get_definition
raise DefinitionNotFound(kw, name)
yangson.exceptions.DefinitionNotFound: grouping mygrouping
Maybe it is a known limitation but I could not find any listing of them.
Best regards,
Iwan
Similarly to #37, yangson considers floats to be valid values for integer type nodes.
Shamelessly stealing the minimal example from the linked issue:
$ cat test.json
{
"ietf-yang-library:modules-state": {
"module-set-id": "",
"module": [
{
"name": "test",
"revision": "1.0.0",
"conformance-type": "implement"
}
]
}
}
$ cat test.yang
module test {
prefix ;
revision 1.0.0;
description "Test schema.";
leaf number {
type int32;
}
}
I would expect the following code to raise an exception, but it doesn't:
from yangson import DataModel
import yaml
data = DataModel.from_file("./test.json")
yaml_input = yaml.safe_load("'test:number': 3.14")
inst = data.from_raw(yaml_input)
inst.validate()
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.