Code Monkey home page Code Monkey logo

ygot's Introduction

Go Coverage Status Go releases supported Go Reference

#ygot

Introduction

ygot (YANG Go Tools) is a collection of Go utilities that can be used to:

  • Generate a set of Go structures and enumerated values for a set of YANG modules, with associated helper methods.
  • Validate the contents of the Go structures against the YANG schema (e.g., validating range and regular expression constraints).
  • Render the Go structures to an output format - such as JSON, or a set of gNMI Notifications for use in a deployment of streaming telemetry.

Whilst ygot is designed to work with any YANG module, for OpenConfig modules, it can provide transformations of the schema to optimise the data structures that are produced for use in systems that generate data instances of the models for configuration purposes. These helper methods require that the OpenConfig style guide patterns are implemented, a model can be verified to conform with these requirements using the OpenConfig linter.

Note: This is not an official Google product.

Getting Started with ygot

Current support for ygot is for the latest 3 Go releases.

ygot consists of a number of parts, generator which is a binary using the ygen library to generate Go code from a set of YANG modules. ygot which provides helper methods for the ygen-produced structs - for example, rendering to JSON, or gNMI notifications - and ytypes which provides validation of the contents of ygen structs against the YANG schema.

The basic workflow for working with ygot is as follows:

  • Generate Go code from a set of YANG files.
  • Write code that populates the Go structures.
  • Validate the contents of the Go structures.
  • Output the contents of the structures as JSON or gNMI Notifications.

The demo/getting_started directory walks through this process for a simple implementation of openconfig-interfaces.

Generating Go Structures from YANG

The generator binary takes a set of YANG modules as input and outputs generated code. For example:

generator -output_file=<outputpath> -package_name=<pkg> [yangfiles]

Will output generated Go code for yangfiles (a space separated list of YANG files) to a file at <outputpath> with the Go package named <pkg>.

Most YANG modules include other modules. If these included modules are not within the current working directory, the path argument is used. The argument to path is a comma-separated list of directories which will be recursively searched for included files.

By default, ygot does not output an entity for the root of the schema tree - such that there is not a root entity to consider in code. If one is desired then it can be produced by using the generate_fakeroot argument. If specified an element with the name specified by fakeroot_name will be created in the output code. By default the fake root element is called device, since the root is often considered to be a device within the OpenConfig use case.

If schema transformations for OpenConfig are desired, these are enabled using the compress_paths argument.

Putting this all together, a command line to generate OpenConfig interfaces from the contents of the demo/getting_started/yang directory is:

go run $GOPATH/src/github.com/openconfig/ygot/generator/generator.go -path=yang -output_file=pkg/ocdemo/oc.go -package_name=ocdemo -generate_fakeroot -fakeroot_name=device -compress_paths=true -shorten_enum_leaf_names -typedef_enum_with_defmod -exclude_modules=ietf-interfaces yang/openconfig-interfaces.yang

To allow this file to be auto-created, you can place a command which allows this code generation to be done automatically, either by creating a file within the YANG directory, or directly embedding this command within the source file that populates the structures. For an example, see the demo/getting_started/main.go file which includes:

//go:generate go run ../../generator/generator.go -path=yang -output_file=pkg/ocdemo/oc.go -package_name=ocdemo -generate_fakeroot -fakeroot_name=device -compress_paths=true -shorten_enum_leaf_names -typedef_enum_with_defmod -exclude_modules=ietf-interfaces yang/openconfig-interfaces.yang

This means that we can simply type go generate within demo/getting_started - and the demo/getting_started/pkg/ocdemo/oc.go is created with the code bindings for the OpenConfig interfaces module.

Writing Code that Populates the Go Structures

Once we have generated the Go bindings for the YANG module, we're ready to use them in an application.

First, let's take a look at what the demo/getting_started/pkg/ocdemo/oc.go file contains. Particularly, looking at the fake root entity that we created (named device):

// Device represents the /device YANG schema element.
type Device struct {
        Interface       map[string]*Interface   `path:"interfaces/interface" rootname:"interface" module:"openconfig-interfaces"`
}

Since we enabled compress_paths, then the /interfaces/interface element in OpenConfig was represented as Interface at the root (called Device). We can see that since interface is a list, keyed by the name element, then the Interface map is keyed by a string.

Looking further down the tree at Interface:

// Interface represents the /openconfig-interfaces/interfaces/interface YANG schema element.
type Interface struct {
        AdminStatus  E_OpenconfigInterfaces_Interface_AdminStatus `path:"state/admin-status" module:"openconfig-interfaces"`
        Counters     *Interface_Counters                          `path:"state/counters" module:"openconfig-interfaces"`
        Description  *string                                      `path:"config/description" module:"openconfig-interfaces"`
        Enabled      *bool                                        `path:"config/enabled" module:"openconfig-interfaces"`
        HoldTime     *Interface_HoldTime                          `path:"hold-time" module:"openconfig-interfaces"`
        Ifindex      *uint32                                      `path:"state/ifindex" module:"openconfig-interfaces"`
        LastChange   *uint32                                      `path:"state/last-change" module:"openconfig-interfaces"`
        Mtu          *uint16                                      `path:"config/mtu" module:"openconfig-interfaces"`
        Name         *string                                      `path:"config/name|name" module:"openconfig-interfaces"`
        OperStatus   E_OpenconfigInterfaces_Interface_AdminStatus `path:"state/oper-status" module:"openconfig-interfaces"`
        Subinterface map[uint32]*Interface_Subinterface           `path:"subinterfaces/subinterface" module:"openconfig-interfaces"`
        Type         E_IETFInterfaces_InterfaceType               `path:"config/type" module:"openconfig-interfaces"`
}

Since OpenConfig path compression was enabled, then this Interface struct contains both direct descendants of /interfaces/interface - such as hold-time (in the Hold-Time field), along with those that were within the config and state fields. The path information is retained in the path struct tag -- but this isn't of interest to most developers working directly with the structs!

We can populate an interface by using a mixture of the helper methods, and directly setting fields of the struct. To create a new interface within the device, we can use the NewInterface method. A New... method is created for all lists within the YANG schema, and takes an argument of the key that is used for the list. It creates a new entry in the map with the specified key, returning an error if the key is already defined.

An example is shown below:

// Create a new interface called "eth0"
i, err := d.NewInterface("eth0")

// Set the fields that are within the struct.
i.AdminStatus = oc.OpenconfigInterfaces_Interface_AdminStatus_UP
i.Mtu = ygot.Uint16(1500)
i.Description = ygot.String("An Interface")

The ygot package provides helpers that allow an input type to returned as a pointer to be populated within the structs. For example, ygot.String returns a string pointer to the argument supplied.

Equally, we can define a new interface directly and add it to the map, without using the NewInterface method:

d.Interface["eth1"] = &oc.Interface{
	Name:        ygot.String("eth1"),
	Description: ygot.String("Another Interface"),
	Enabled:     ygot.Bool(false),
	Type:        oc.IETFInterfaces_InterfaceType_ethernetCsmacd,
}

Validating the Struct Contents

For some fields of the structures, enumerated values for example, values of fields are restricted such that they cannot have invalid values specified. In other cases, such as an IPv4 addresses, a string may not match a regular expression, but the Go structure does not restrict the contents of the struct being populated with this data.

By default each struct has a Validate method, this can be used to validate the struct's contents against the schema. Validate can be called against each structure, for example:

if err := d.Interface["eth0"].Validate(); err != nil {
	panic(fmt.Sprintf("Interface validation failed: %v", err))
}

In the case that the struct does not contain valid contents, Validate returns an error, containing a list of errors encountered during validation of the struct contents. Whilst the error can be directly handled as a comma-separated list of strings containing validation errors, casting it to the ytypes.Errors type allows handling of individual errors more cleanly. For example:

_, err = subif.Ipv4.NewAddress("Not a valid address")
if err := invalidIf.Validate(); err == nil {
	panic(fmt.Sprintf("Did not find invalid address, got nil err: %v", err))
} else {
	errs := err.(ytypes.Errors)
	for _, err := range errs {
		fmt.Printf("Got expected error: %v\n", err)	}
}

Outputting JSON from GoStructs

To serialise the structures to JSON, the ygot package provides an EmitJSON method which can be called with an arbitrary structure. In the example below, the fake root (Device) struct is called:

json, err := ygot.EmitJSON(d, &ygot.EmitJSONConfig{
	Format: ygot.RFC7951,
	Indent: "  ",
	RFC7951Config: &ygot.RFC7951JSONConfig{
		AppendModuleName: true,
 	},
})

if err != nil {
	panic(fmt.Sprintf("JSON demo error: %v", err))
}
fmt.Println(json)

EmitJSON performs both Validate and outputs the structure to JSON. The format can be an internal JSON format, or that described by RFC7951. Validation or JSON marshalling errors are directly returned.

Unmarshalling JSON to a GoStruct

ygot includes a function to unmarshal data from RFC7951-encoded JSON to a GoStruct. Since this function relies on the schema of the generated code, it us output within the generated code package - and named Unmarshal. The function takes an argument of a []byte (byte slice) containing the JSON document to be unmarshalled, and a pointer to the struct into which it should be unmarshalled. Any struct can be unmarshalled into. If data cannot be unmarshalled, an error is returned.

To unmarshal the example created in this guide, we call Unmarshal with the oc.Device struct pointer, and the JSON document:

// Device struct to unmarshal into.
loadd := &oc.Device{}
if err := oc.Unmarshal([]byte(json), loadd); err != nil {
  panic(fmt.Sprintf("Cannot unmarshal JSON: %v", err))
}

Currently, only the RFC7951 format of JSON is supported for unmarshalling, the Internal format supported by ygot is not yet supported.

Interacting with gNMI Server(s) via ygot GoStructs

While ygot provides Go types for structuring YANG data, the process of interacting with a gNMI server is often cumbersome:

  1. Marshalling a GoStruct to JSON.
  2. Creating a gNMI SetRequest or SubscribeRequest object.
  3. Processing the SubscribeResponse stream and accumulating per-path errors.
  4. Unmarshalling the response from scalar protobufs or JSON to a GoStruct.

If this applies to you, refer to ygnmi for a ygot-based tool that generates not only the same ygot GoStructs, but also helpers for each GoStruct and leaf example code. ygnmi is used by Ondatra to create its gNMI test library used in [featureprofiles](This too://github.com/openconfig/featureprofiles).

For Developers

  • Contributing - how to contribute to ygot.
  • Contributors - Folks who have contributed to ygot, thanks very much!
  • Design Choices - This document provides information pertaining to design choices made within the library itself; and should be reviewed in conjunction with the comments in the library code.

Licensing

Copyright 2017 Google Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

ygot's People

Contributors

ankur19 avatar armanhb avatar chentianheng604 avatar clubfest avatar dan-lepage avatar dang100 avatar davidkorczynski avatar dependabot[bot] avatar dkarwowski avatar dsnet avatar g-yusufsn avatar greg-dennis avatar hansthienpondt avatar hellt avatar idefixcert avatar idryzhov avatar jayzhudev avatar k1eran avatar kjahed avatar lgomez9 avatar luukp avatar mkhsueh avatar n0shut avatar ostromart avatar robshakir avatar seancondon avatar shichuzhu avatar soumiksamanta avatar steiler avatar wenovus avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ygot's Issues

unmarshaling JSON

Hi,
I got unexpected field ipv4-acl

"F0121 22:30:19.558646 2294 gnmi_set.go:158] Set failed: rpc error: code = InvalidArgument desc = unmarshaling json data to config struct fails: parent container device (type *ocstruct.Device): JSON contains unexpected field ipv4-acl"

I did compile YANG to go and in the go file the struct is

type Device struct {
	Hostname	*string	`path:"hostname" module:"faucet-configuration"`
	Ipv4Acl	*FaucetConfiguration_Ipv4Acl	`path:"ipv4-acl" module:"faucet-configuration"`
}

I got this error when I tried to send JSON config using gnmi_set

{
  "hostname":"raspberrypi",
  "ipv4-acl":{
   "destination-ipv4-network": "192.168.1.0/24"
  }
}

I also tried with IETF JSON formate, "(faucet-configuration:ipv4-acl)"it gives same error.

YANG module is

module faucet-configuration {

  yang-version "1";

  // namespace
  namespace "http://github.com/faucetsdn";

  prefix "faucet-conf";

  import ietf-inet-types {
    prefix "inet";
  }
// from https://github.com/YangModels/yang/tree/master/experimental/ietf-extracted-YANG-modules
 import ietf-packet-fields {
    prefix packet-fields;
  }

  // meta
  organization "faucet, Inc.";

  contact
    "non";

  description
    "This module is an experimental module.";

  revision "2018-01-22" {
    description
      "Initial version";
    reference "0.0.1";
  }

  grouping faucet-top {
    description
      "top level";

    leaf hostname {
      type inet:host;
      description
        "faucet host";
    }

  container ipv4-acl {
      uses packet-fields:acl-ipv4-header-fields;
      description
           "IPv4 headers.";
      }
  }

  uses faucet-top;
}

Automatically generate a FindOrNew function for every list

I see a bit of an anti pattern forming in my ygot code. One function that works with a port calls NewInterface and future functions must check for the existence of the list element.

Example:
port, err := dev.NewInterface(portName)
if err != nil {
return err
}

Next function that works with this interface:
port := dev.Interface[portName]
if port == nil {
port, err = dev.NewInterface(portName)
if err != nil {
return err
}
}

It would be nice if there was a helper function that would allow each caller to do:

port := dev.FindOrNewInterface(portName)

This is very similar to the Google map_util.h helper functions.
https://github.com/google/protobuf/blob/master/src/google/protobuf/stubs/map_util.h

If I have time, I might try adding this......

Additional data types for leafref validations.

  • leaf-list containing leafref.
  • typedef that defines a leafref.
  • (for Y1.1) union that defines a leafref.

// leaf-list of leafref with a target of a leaf-ref
leaf-list foo {
type leafref {
path "../another-leaflist"
}
}

leaf-list another-leaflist {
type string;
}

"another-leaflist": ["one", "two", "three"]
"foo": ["one", "three"] // must be a subset of the other leafref.

// leaf-list of leafref with a target of a list key
leaf-list foo {
type leafref {
path "../a-list/key";
}
}

list a-list {
key "key";
leaf key {
type string;
}
}

"a-list": [{"key": "one"}, {"key": "two"}]
"foo": ["one"] // must be a subset of the keys of the target list

// leaf-list of leafref with a target of a scalar leaf (probably not useful :-))
leaf-list foo {
type leafref {
path "../bar";
}
}

leaf bar {
type string;
}

"bar": "one",
"foo": ["one"] // must be equal to the target leaf, but useless to have this construct because leaf-list can only have one member

Protobuf fields could be grouped by type.

In order that fields that have their type defined by the same module (e.g., openconfig.enums.FOO and openconfig.enums.BAR) are grouped together, field sorting could by via type rather than name. This needs post-processing of the fields since the type is not known at creation time. It could be done in genProto3MsgCode.

Refactor path storage in ygot

Today, the path struct tag stores a path relative to the parent struct that the field is within - i.e., As has the path config/as in the Bgp_Global struct, other than root entities (e.g., Bgp) which does not always have a parent, and hence has bgp/global as the path for Bgp_Global. This results in special case handling being required for root-level structs.

Proposed plan, from offline discussion:

  • Make the path that is stored in the path struct tag absolute in all cases.
  • Fix ygen findMapPaths such that it outputs the absolute path rather than the relative one. This is unrolling this code.
  • Fix ygot structPathToLibPaths to deduplicate based on the parentPath supplied to them. Move this call to a shared library between ytypes and ygot.
  • Change pathToSchema in ytypes to use this approach to retrieve the path from a struct tag.

TogNMINotifications should adopt gNMI 0.4.0 Paths.

TogNMINotifications currently uses pre-0.4.0 paths, of the form:

p := &gnmi.Path{
  Element: []string{"interfaces", "interface", "eth0"},
}

In gNMI 0.4.0 then these should be of the form:

p := &gnmi.Path{
  Elem: []*gnmipb.PathElem{{
      Name: "interfaces",
  }, {
      Name: "interface",
      Key: map[string]string{"name": "eth0"},
 }},
}

Default value of a enum is not used as the UNSET value in Go

In the protobuf generation code, the default value of an enumeration is read and this is used as the UNSET value - such that the default value of the enum is the default. This same logic is not implemented in the Go code - but should be.

Proto Enum Naming could be more aesthetically pleasing.

Today we implement the following pattern for enum naming (global enums):

package package_name.enums;

enum YangModuleNameENUMNAMEWITHNOUNDERSCORES {
  YANGMODULENAME_ENUMNAMEWITHNOUNDERSCORES_UPPERCASE_VALUE_NAME = 1;
}

The proposal is to adjust this to be:

package package_name.YangModuleName;  // this package already exists since it's where the nested messages are output.

enum EnumNameInCamelCase {
  ENUM_NAME_WITH_UNDERSCORES_UPPERCASE_VALUE_NAME = 1;
}

We need the ENUM_NAME_WITH_UNDERSCORES since proto->C++ requires that we have a unique prefix on each enumerated value's name.

Move path functions to separate file/package.

Path processing functions are now scattered over several files. It makes sense to group them into either a separate package (if they are useful as a separate entity externally) or at least a separate file.

goyang schema tree does not include module prefixes

This means in principle that keys from different modules may overlap in the schema tree. This could happen if (say) two augments from different modules share the same key name.
Since this is an unlikely scenario, this issue is just to raise awareness for users that this could happen. It's unclear that the effort required to fix it in goyang and dependencies is worth it.

Validate does not verify references.

Validate currently has no way of traversing non-descendant nodes in the data tree. Once this facility is available it should verify references.

De-duplicate Identical Struct Types in Generated Code

In some schemas, e.g., the openconfig-bgp module, we have identical structs produced for NetworkInstance_Protocol_Bgp_Neighbor_AfiSafi is identical to NetworkInstance_Protocol_Bgp_PeerGroup_AfiSafi - currently ygen produces these two types as different structs - this makes handling code need to either use reflection, or creates some duplication for two functions that handle different types.

ygen could determine for the list of structs defined whether they are common, and then subsequently, deduplicate them. This is not the same as creating code for groupings, since each struct may consist of multiple groupings.

A consideration of this approach is that during adding augmentations the types may change. Consider the following scenario:

module a {
  container one {
     leaf string { type string; }
  }
  container two {
     leaf string { type string; }
  }
}

In this case, current code generation would produce a A_One and A_Two struct, and deduplicate them such that the same type is used - say A_OneTwo.

If we then introduce a second module:

module b {
  import a { prefix a; }
  
  augment /a:one {
     leaf another-string { type string; }
  }
}

Then deduplication would not be possible between /one and /twoand hence, the types would need to split again. Users that useGetOrCreateXXX` methods would be insulated from such changes, but those that define struct literals will need to consider such changes.

The proposal is to implement this deduplication.

when creating a keyed list entry, the reference node of the key should be created first

In the generated go struct code:

func (t *OpenconfigPlatform_Components) NewComponent(Name string) (*OpenconfigPlatform_Components_Component, error)

creates a OpenconfigPlatform_Components_Component object with the specified name. However, name is a ref leaf, so the underlying config node should also be created.

The object created by NewComponent() fails on validation.

ygot puts the branch key to the top-level model by mistake.

We have a model defined as the following: https://github.com/google/link022/blob/da4123fbada06e7a4729b0f8841c96fdf66dae79/openconfig/models/wifi-office.yang

Here is the generated top-level model:
// Office represents the /office YANG schema element.
type Office struct {
Hostname *string path:"hostname" module:"openconfig-office-ap" <--- ISSUE
OfficeAp map[string]*WifiOffice_OfficeAp path:"office-ap" module:"wifi-office"
OfficeName *string path:"office-name" module:"wifi-office"
Vendor map[string]*WifiOffice_Vendor path:"vendor" module:"wifi-office"
VendorConfig map[string]*WifiOffice_VendorConfig path:"vendor-config" module:"wifi-office"
}

The hostname is defined inside OfficeAP, which is the key of map.

Leafref with path specifying current() does not resolve correctly.

When a leafref contains current() then the nil set is found rather than existing leaves. See 5f05e49

Output of failing test:

[16:25] ygot/ytypes/schema_tests   leafref_current ✔                                                                    5m ◒  
▶ go test
--- FAIL: TestLeafrefCurrent (0.03s)
	validate_test.go:838: TestLeafrefCurrent: could not validate populated interfaces, got: pointed-to value with path /interfaces/name=current()/../interface]/subinterfaces/subinterface/index from field Subinterface value 0 (uint32 ptr) schema /device/mpls/global/interface-attributes/interface/interface-ref/config/subinterface is empty set, want: nil
FAIL
exit status 1
FAIL	github.com/openconfig/ygot/ytypes/schema_tests	0.723s

Support YANG empty type

The YANG empty type is represented as a bool in the output structs, but should be serialised into JSON as [null]. Since we want to keep empty represented as a bool, then this needs some additional work, either in the way of an annotation for these leaves, or a derived type that can be matched against like binary.

Validate with LeafrefOptions should allow a path mask to be specified.

Currently, LeafrefOptions allows missing references to be ignored during a Validate call. For example, where a leafref references some entity that is missing. However, it does this over the whole tree - essentially turning off all reference validation.

Some callers may know the subset of the tree that they expect to be populated - e.g., a method that is populating all of the BGP subtree may expect peer groups to exist, but doesn't know that routing policy has been populated. Given that this is the case, we should extend the LeafrefOptions struct to take a set of path expressions that specify a mask for the paths that are validated.

The API to this feature is suggested to look something like the following:

d := &oc.Device{}
// populate d.
if err := d.Validate(&ytypes.LeafrefOpts{Paths: []string{"/network-instances/network-instance/protocols/bgp.*"}}); err != nil {
  // handle error
}

Where the Paths field is a string slice containing a set of regular expressions which match gNMI path format strings.

invalid module name prefix in IETF JSON

ygot.ConstructIETFJSON() can generate the following JSON config:

{
"openconfig-system:openconfig-openflow:system": {
"openflow": {
"agent": {
"config": {
"backoff-interval": 5,
"datapath-id": "00:16:3e:00:00:00:00:00",
"failure-mode": "SECURE",
"inactivity-probe": 10,
"max-backoff": 10
}
}
}
},
"openconfig-system:system": {
"clock": {
"config": {
"timezone-name": "Europe/Stockholm"
}
},
"config": {
"domain-name": "google.com",
"hostname": "switch_a",
"login-banner": "Hello!",
"motd-banner": "Hi There!"
}
}
}

The first "system" section contains more than one module name as a prefix.
However, according toRFC 7951 section 4 “Names and Namespaces”, this is invalid:
there can only be a single module name preceding the element name.

Path compression in schema causes lookup complexity/bugs.

@ostromart

Let's scheme on this next week. Currently unmarshalling /components/component/state/type fails, since because it contains two identityref types within a union. The lookup that we get to is /component/state/type because calling Path() on the schema tree yang.Entry gives the path relative to the fakeroot. This means that we have the same issue where root-level containers that are compressed out of the schema are not reflected in the paths.

There are two fixes for this - one is demonstrated in #84. This approach says never to use Path() on the returned yang.Entry in the schematree, and rather use the schemapath annotation that's added there. This is a pretty kludgy fix.

The alternate approach is that we create a faithful representation of the schematree as goyang would give us. This means some changes:

  • The schematree could include the modules as root level yang.Entry elements.
  • Some entities that were previously compressed out at the root would be reinjected.

Looking at the code, this seems like it'd only really affect the root level entities/handling of the fakeroot. I think it's OK to handle in ygen -- but it seems like this has caused us enough trips that we should probably re-examine at this point.

Add flag to allow read-only fields to be skipped in code generation.

For some use cases (e.g., configuration generation) - there is no requirement for config false fields to be included in generated structs. To keep the generated code size down, and aid comprehension by developers working with the generated code, we should implement an option to skip generation of fields in ygen.

This should be agnostic to the output format.

Mechanism to access default values

It is common for a leaf node to have a default value (here is an example). If those nodes are unset, the current behavior is that the node is marked as unset (nil). This is not ideal as the Go code lost the information of the default value. I can think of two possible solutions to this.

  1. Provide an additional API for each node to access its default value (nil if there is no default value). Or
  2. Set the field to have the value of the default even if the field is unset.

Option 2) has the drawback of losing the fact when a field is purposefully unset but either implementation would be a good step forward compared with the current behavior.

Duplicated struct generation

When generating golang structures with the following yang model, generator generates two golang structs, Top_A_C and Top_B_C which corresponds to container c despite the fields inside them are identical.

Is it possible to suppress this duplication?

module test {
    yang-version 1;
    namespace "http://example.com";
    prefix "test";

    grouping g {
        container c {
            leaf value {
                type uint8;
            }
        }
    }

    container top {
        container a {
            uses g;
        }

        container b {
            uses g;
        }
    }

}

Proto3 generation should support leaf-list of union

Currently leaf-list of union will not function as a oneof cannot be repeated. In this case, we should probably create a message that corresponds to the oneof, and have this repeated in the parent message.

Couldn't get correct leafref which under the list

Hi, guys, I tried the basic workflow of ygot for openconfig-telemetry yang model, but it fails to validate the contents of Go struct. I think the the contents of struct is complete. It looks like validate module couldn't get correct leafref which under the list.

Here is the schema tree and struct contents.

telemetry-system (container)
  destination-groups (container)
    destination-group (list)
      config (container)
        group-id (leaf)
      destinations (container)
        destination (list)
          config (container)
            destination-address (leaf)
            destination-port (leaf)
          destination-address (leaf)
          destination-port (leaf)
          state (container)
            destination-address (leaf)
            destination-port (leaf)
      group-id (leaf)
      state (container)
        group-id (leaf)
  sensor-groups (container)
    sensor-group (list)
      config (container)
        sensor-group-id (leaf)
      sensor-group-id (leaf)
      sensor-paths (container)
        sensor-path (list)
          config (container)
            exclude-filter (leaf)
            path (leaf)
          path (leaf)
          state (container)
            exclude-filter (leaf)
            path (leaf)
      state (container)
        sensor-group-id (leaf)
  subscriptions (container)
    persistent (container)
      subscription (list)
        config (container)
          encoding (leaf)
          local-source-address (leaf)
          originated-qos-marking (leaf)
          protocol (leaf)
          subscription-name (leaf)
        destination-groups (container)
          destination-group (list)
            state (container)
              group-id (leaf)
            config (container)
              group-id (leaf)
            group-id (leaf)
        sensor-profiles (container)
          sensor-profile (list)
            config (container)
              heartbeat-interval (leaf)
              sample-interval (leaf)
              sensor-group (leaf)
              suppress-redundant (leaf)
            sensor-group (leaf)
            state (container)
              heartbeat-interval (leaf)
              sample-interval (leaf)
              sensor-group (leaf)
              suppress-redundant (leaf)
        state (container)
          protocol (leaf)
          subscription-id (leaf)
          subscription-name (leaf)
          encoding (leaf)
          local-source-address (leaf)
          originated-qos-marking (leaf)
        subscription-name (leaf)
    dynamic (container)
      subscription (list)
        sensor-paths (container)
          sensor-path (list)
            path (leaf)
            state (container)
              exclude-filter (leaf)
              path (leaf)
        state (container)
          encoding (leaf)
          subscription-id (leaf)
          sample-interval (leaf)
          suppress-redundant (leaf)
          destination-address (leaf)
          destination-port (leaf)
          heartbeat-interval (leaf)
          originated-qos-marking (leaf)
          protocol (leaf)
        subscription-id (leaf)
     [device (container)]
      TelemetrySystem [telemetry-system (container)]
        DestinationGroups [destination-groups (container)]
          DestinationGroup [destination-group (list)]
          HS
            Config [config (container)]
                GroupId : "HS" [group-id (leaf)]
            Destinations [destinations (container)]
              Destination [destination (list)]
              {20.20.20.20 2345}
                Config [config (container)]
                    DestinationAddress : "20.20.20.20" [destination-address (leaf)]
                    DestinationPort : 2345 [destination-port (leaf)]
                  DestinationAddress : "20.20.20.20" [destination-address (leaf)]
                  DestinationPort : 2345 [destination-port (leaf)]
                State [state (container)]
                    DestinationAddress : "20.20.20.20" [destination-address (leaf)]
                    DestinationPort : 2345 [destination-port (leaf)]
              GroupId : "HS" [group-id (leaf)]
        SensorGroups [sensor-groups (container)]
          SensorGroup [sensor-group (list)]
          HS_PORT_COUNTERS
            Config [config (container)]
                SensorGroupId : "HS_PORT_COUNTERS" [sensor-group-id (leaf)]
              SensorGroupId : "HS_PORT_COUNTERS" [sensor-group-id (leaf)]
            SensorPaths [sensor-paths (container)]
              SensorPath [sensor-path (list)]
              /counters/pfc
                Config [config (container)]
                    Path : "/counters/pfc" [path (leaf)]
                  Path : "/counters/pfc" [path (leaf)]
                State [state (container)]
                    Path : "/counters/pfc" [path (leaf)]
            State [state (container)]
                SensorGroupId : "HS_PORT_COUNTERS" [sensor-group-id (leaf)]
        Subscriptions [subscriptions (container)]
                  Encoding : 0 [encoding (leaf)]
                  Protocol : 0 [protocol (leaf)]
          Persistent [persistent (container)]
            Subscription [subscription (list)]
            HS_PORT
              Config [config (container)]
                  Encoding : 1 [encoding (leaf)]
                  LocalSourceAddress : "10.10.10.10" [local-source-address (leaf)]
                  Protocol : 1 [protocol (leaf)]
                  SubscriptionName : "HS_PORT" [subscription-name (leaf)]
              DestinationGroups [destination-groups (container)]
                DestinationGroup [destination-group (list)]
                HS
                  Config [config (container)]
                      GroupId : "HS" [group-id (leaf)]
                    GroupId : "HS" [group-id (leaf)]
                  State [state (container)]
                      GroupId : "HS" [group-id (leaf)]
              SensorProfiles [sensor-profiles (container)]
                SensorProfile [sensor-profile (list)]
                HS_PORT_COUNTERS
                  Config [config (container)]
                      HeartbeatInterval : 30000 [heartbeat-interval (leaf)]
                      SampleInterval : 1000 [sample-interval (leaf)]
                      SensorGroup : "HS_PORT_COUNTERS" [sensor-group (leaf)]
                      SuppressRedundant : true [suppress-redundant (leaf)]
                    SensorGroup : "HS_PORT_COUNTERS" [sensor-group (leaf)]
                  State [state (container)]
                      HeartbeatInterval : 30000 [heartbeat-interval (leaf)]
                      SampleInterval : 1000 [sample-interval (leaf)]
                      SensorGroup : "HS_PORT_COUNTERS" [sensor-group (leaf)]
                      SuppressRedundant : true [suppress-redundant (leaf)]
              State [state (container)]
                  Encoding : 1 [encoding (leaf)]
                  LocalSourceAddress : "10.10.10.10" [local-source-address (leaf)]
                  Protocol : 1 [protocol (leaf)]
                  SubscriptionName : "HS_PORT" [subscription-name (leaf)]
                SubscriptionName : "HS_PORT" [subscription-name (leaf)]

"openconfig.OpenconfigTelemetry_TelemetrySystem_Subscriptions_Persistent_Subscription_DestinationGroups_DestinationGroup_Config" should get the root path node "openconfig.OpenconfigTelemetry_TelemetrySystem" not "openconfig.OpenconfigTelemetry_TelemetrySystem_Subscriptions_Persistent".

DataNodeAtPath got leafref with path elem:<name:".." > elem:<name:".." > elem:<name:".." > elem:<name:".." > elem:<name:".." > elem:<name:".." > elem:<name:".." > elem:<name:"destination-groups" > elem:<name:"destination-group" > elem:<name:"group-id" >  from node path /device/telemetry-system/subscriptions/persistent/subscription/destination-groups/destination-group/config/group-id, field name GroupId
going up data tree from type *string to *openconfig.OpenconfigTelemetry_TelemetrySystem_Subscriptions_Persistent_Subscription_DestinationGroups_DestinationGroup_Config, schema path from parent is [group-id], remaining path elem:<name:".." > elem:<name:".." > elem:<name:".." > elem:<name:".." > elem:<name:".." > elem:<name:".." > elem:<name:"destination-groups" > elem:<name:"destination-group" > elem:<name:"group-id" > 
going up data tree from type *openconfig.OpenconfigTelemetry_TelemetrySystem_Subscriptions_Persistent_Subscription_DestinationGroups_DestinationGroup_Config to *openconfig.OpenconfigTelemetry_TelemetrySystem_Subscriptions_Persistent_Subscription_DestinationGroups_DestinationGroup, schema path from parent is [config], remaining path elem:<name:".." > elem:<name:".." > elem:<name:".." > elem:<name:".." > elem:<name:".." > elem:<name:"destination-groups" > elem:<name:"destination-group" > elem:<name:"group-id" > 
going up data tree from type *openconfig.OpenconfigTelemetry_TelemetrySystem_Subscriptions_Persistent_Subscription_DestinationGroups_DestinationGroup to map[string]*openconfig.OpenconfigTelemetry_TelemetrySystem_Subscriptions_Persistent_Subscription_DestinationGroups_DestinationGroup, schema path from parent is [destination-group], remaining path elem:<name:".." > elem:<name:".." > elem:<name:".." > elem:<name:".." > elem:<name:"destination-groups" > elem:<name:"destination-group" > elem:<name:"group-id" > 
going up data tree from type map[string]*openconfig.OpenconfigTelemetry_TelemetrySystem_Subscriptions_Persistent_Subscription_DestinationGroups_DestinationGroup to *openconfig.OpenconfigTelemetry_TelemetrySystem_Subscriptions_Persistent_Subscription_DestinationGroups, schema path from parent is [destination-group], remaining path elem:<name:".." > elem:<name:".." > elem:<name:".." > elem:<name:"destination-groups" > elem:<name:"destination-group" > elem:<name:"group-id" > 
going up data tree from type *openconfig.OpenconfigTelemetry_TelemetrySystem_Subscriptions_Persistent_Subscription_DestinationGroups to *openconfig.OpenconfigTelemetry_TelemetrySystem_Subscriptions_Persistent_Subscription, schema path from parent is [destination-groups], remaining path elem:<name:".." > elem:<name:".." > elem:<name:"destination-groups" > elem:<name:"destination-group" > elem:<name:"group-id" > 
going up data tree from type *openconfig.OpenconfigTelemetry_TelemetrySystem_Subscriptions_Persistent_Subscription to map[string]*openconfig.OpenconfigTelemetry_TelemetrySystem_Subscriptions_Persistent_Subscription, schema path from parent is [subscription], remaining path elem:<name:".." > elem:<name:"destination-groups" > elem:<name:"destination-group" > elem:<name:"group-id" > 
going up data tree from type map[string]*openconfig.OpenconfigTelemetry_TelemetrySystem_Subscriptions_Persistent_Subscription to *openconfig.OpenconfigTelemetry_TelemetrySystem_Subscriptions_Persistent, schema path from parent is [subscription], remaining path elem:<name:"destination-groups" > elem:<name:"destination-group" > elem:<name:"group-id" > 
root element type *openconfig.OpenconfigTelemetry_TelemetrySystem_Subscriptions_Persistent with remaining path elem:<name:"destination-groups" > elem:<name:"destination-group" > elem:<name:"group-id" > 
. GetNode next path name:"destination-groups" , value { map[HS_PORT:0xc4202bdf50] (map ptr) }
. getNodesContainer: schema persistent, next path name:"destination-groups" , value { map[HS_PORT:0xc4202bdf50] (map ptr) }
. check field name subscription, paths [[subscription]]
. ERR: could not find path in tree beyond schema node persistent, (type *openconfig.OpenconfigTelemetry_TelemetrySystem_Subscriptions_Persistent), remaining path elem:<name:"destination-groups" > elem:<name:"destination-group" > elem:<name:"group-id" > 

It looks like get the wrong data type this step

going up data tree from type map[string]*openconfig.OpenconfigTelemetry_TelemetrySystem_Subscriptions_Persistent_Subscription_DestinationGroups_DestinationGroup to *openconfig.OpenconfigTelemetry_TelemetrySystem_Subscriptions_Persistent_Subscription_DestinationGroups, schema path from parent is [destination-group], remaining path elem:<name:".." > elem:<name:".." > elem:<name:".." > elem:<name:"destination-groups" > elem:<name:"destination-group" > elem:<name:"group-id" > 

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.