nautobot / nautobot-app-nornir Goto Github PK

View Code? Open in Web Editor NEW

24.0 24.0 17.0 1.74 MB

Nornir plugin to enable other Nautobot network automation plugins

Home Page: https://docs.nautobot.com/projects/plugin-nornir/en/latest/

License: Other

Dockerfile 6.03% Python 93.97%

nautobot nautobot-plugin

nautobot-app-nornir's Introduction

Nautobot

Nautobot is a Network Source of Truth and Network Automation Platform built as a web application atop the Django Python framework with a PostgreSQL or MySQL database.

Key Use Cases

1. Flexible Source of Truth for Networking - Nautobot core data models are used to define the intended state of network infrastructure enabling it as a Source of Truth. While a baseline set of models are provided (such as IP networks and addresses, devices and racks, circuits and cable, etc.) it is Nautobot's goal to offer maximum data model flexibility. This is enabled through features such as user-defined relationships, custom fields on any model, and data validation that permits users to codify everything from naming standards to having automated tests run before data can be populated into Nautobot.

2. Extensible Data Platform for Automation - Nautobot has a rich feature set to seamlessly integrate with network automation solutions. Nautobot offers GraphQL and native Git integration along with REST APIs and webhooks. Git integration dynamically loads YAML data files as Nautobot config contexts. Nautobot also has an evolving plugin system that enables users to create custom models, APIs, and UI elements. The plugin system is also used to unify and aggregate disparate data sources creating a Single Source of Truth to streamline data management for network automation.

3. Platform for Network Automation Apps - The Nautobot plugin system enables users to create Network Automation Apps. Apps can be as lightweight or robust as needed based on user needs. Using Nautobot for creating custom applications saves up to 70% development time by re-using features such as authentication, permissions, webhooks, GraphQL, change logging, etc. all while having access to the data already stored in Nautobot. Some production ready applications include:

The complete documentation for Nautobot can be found at Read the Docs.

Questions? Comments? Start by perusing our GitHub discussions for the topic you have in mind, or join the #nautobot channel on Network to Code's Slack community!

Build Status

Branch	Status
main
develop
next

Screenshots

Installation

Please see the documentation for instructions on installing Nautobot.

Application Stack

Below is a simplified overview of the Nautobot application stack for reference:

Plugins and Extensibility

Nautobot offers the ability to customize your setup to better align with your direct business needs. It does so through the use of various plugins that have been developed for network automation, and are designed to be used in environments where needed.

There are many plugins available within the Nautobot Apps ecosystem. The below screenshots are an example of some popular ones that are currently available.

Plugin Screenshots

Golden Config Plugin

ChatOps Plugin

Device Lifecycle Management Plugin

Providing Feedback

The best platform for general feedback, assistance, and other discussion is our GitHub discussions. To report a bug or request a specific feature, please open a GitHub issue using the appropriate template.

If you are interested in contributing to the development of Nautobot, please read our contributing guide prior to beginning any work.

Related projects

Please check out the GitHub nautobot topic for a list of relevant community projects.

Notices

Nautobot was initially developed as a fork of NetBox (v2.10.4). NetBox was originally developed by Jeremy Stretch at DigitalOcean and the NetBox Community.

nautobot-app-nornir's People

Contributors

Stargazers

Watchers

Forkers

rileyl6122428 jifox itdependsnetworks jpobeda jvanderaa bsmeding chipn baclaeys nanisyako mitchell-foxworth chadell cmsirbu justinbrink alextremblay viehoang68 pawpawzz

nautobot-app-nornir's Issues

Environment variables match Nautobot cookie/dev env variables?

The default environment variables for env credentials type are NAPALM_USER and NAPALM_PASSWORD, should these also accept NAUTOBOT_ prepended as is the default in the various environments for deploying Nautobot and consistency with other environment variables?

Add new credentials integration with Nautobot

Environment

Python version: 3.6.2
Nautobot version: 1.2.1
nautobot-plugin-nornir version: 0.9.7

Proposed Functionality

Implement Nautobot Secrets as a backend option to retrieve secrets

Use Case

Be able to eventually integrate with other backends such as Hashicorp Vault or AWS Secrets Manager

Have a platform mapper to support the different tool mappings

Environment

Python version: 3.7+
Nautobot version: 1.3+
nautobot-plugin-nornir version: 1.0.0

Proposed Functionality

Utilize netutils mapping functions or user defined mappings to tranform the inventory platform for different nornir task plugins.

Use Case

Platform definitions between nautobot, netmiko, scrapli, etc aren't a one to one mapping. If you using nautobot inventory but want to call netmiko or another nornir task, you currently have to go manually update the platform entry in the nornir inventory host object.

Having a easy way to transpose this would be helpful and avoid having different platform mapping functions spread out between different plugins.

Add jinja settings support for golden config plugin

Environment

Python version: 3.10
Nautobot version: 1.6
nautobot-plugin-nornir version: 1.0.0

Proposed Functionality

golden-config#527 involves the ability to pass settings for the jinja environment into nornir-nautobot.

That capability has been added to nornir-nautobot v2.5.0, but nautobot-plugin-nornir still depends on v2.3.0, and nautobot-plugin-golden-config depends on nautobot-plugin-nornir. golden-config#527 is blocked until this can be fixed

Use Case

Nautobot Credential Manager

Environment

Python version: 3.7
Nautobot version: 1.0
nautobot-plugin-nornir version: 0.9

Proposed Functionality

The ability to set credentials from not only environment variables, but instead from the plugin configuration.

Use Case

The credentials can be just as easily stored in nautobot_config.py as it can in environment variables.

Issue #10

Default Nornir settings have different structure than the ones used in Plugin Config

Environment

nautobot-plugin-nornir version:

Expected Behavior

The config used by NORNIR_SETTINGS is consistent either is coming from settings.PLUGINS_CONFIG or via default _NORNIR_SETTINGS, but they look to be different.

Observed Behavior

default settings: https://github.com/nautobot/nautobot-plugin-nornir/blob/develop/nautobot_plugin_nornir/constants.py#L5-L9

_NORNIR_SETTINGS = {
    "inventory": "nautobot_plugin_nornir.plugins.inventory.nautobot_orm.NautobotORMInventory",
    "credentials": "nautobot_plugin_nornir.plugins.credentials.env_vars.CredentialsEnvVars",
    "nornir.core": {"num_workers": 20},
}

Definitive settings are taken from Plugin config "nornir_settings" ( if available: https://github.com/nautobot/nautobot-plugin-nornir/blob/develop/nautobot_plugin_nornir/constants.py#L12
and the data is:

PLUGINS_CONFIG = {
    "nautobot_plugin_nornir": {
        "nornir_settings": {
            "credentials": "nautobot_plugin_nornir.plugins.credentials.env_vars.CredentialsEnvVars",
            "runner": {
                "plugin": "threaded",
                "options": {
                    "num_workers": 20,
                },
            },
        },
    },
}

so, comparing both:

"credentials": "nautobot_plugin_nornir.plugins.credentials.env_vars.CredentialsEnvVars",
"runner": {
   "plugin": "threaded",
       "options": {
            "num_workers": 20,
       },
   },

So, I see a difference between the defaults, using "nornir.core" to define the workers, and the plugin config using "runner"/"options".

For instance, in one example of library usage, in https://github.com/nautobot/nautobot-plugin-golden-config/blob/0c3e902582d1ce68094d731aa614f35fc28e701f/nautobot_golden_config/nornir_plays/config_backup.py#L114 , it is looking for "runner" that is not defined by default in the runner

Maybe I'm missing something?

Using two secrets groups on devices within same platform doesn't work

Environment

Python version: 3.8
Nautobot version: 1.5
nautobot-plugin-nornir version: latest

Steps to Reproduce

Create two devices, eg router and switch, with same platform eg cisco_ios
Create two secrets group. With diff creds.
Assign one group to each device.
Run a shim job (GC) with both devices selected

Expected Behavior

Both devices work and use the secrets info from the group attached to the device object.

Observed Behavior

Each device object has the correct user/pass based on secrets group; however the secret that is created within the extras in configuration_options will both be assigned the "last" object that is looped throughs secret.

From initial testing something with the groups is causing this. In nautobot_orm.py.

            # secrets are populated correctly at this point.
            for group in hosts[device.name].groups:
                if group not in groups.keys():
                    groups[group] = Group(name=group, defaults=defaults)
            # secrets are broken at this point.

Seems like because both devices are part of the same "platform" group, its causing the problem.

Enable get_group_creds

Enable get_group_creds within the inventory.

Provide better Error Handling when credential class isn't provided.

Environment

Python version: 3.7
Nautobot version: 1.2.5
nautobot-plugin-nornir version: Current

Steps to Reproduce

Install plugin and define plugin configuration without a credentials class setting.

PLUGINS_CONFIG = {
    "nautobot_plugin_nornir": {
        "dispatcher_mapping": None,
        "username": "cool_user",
        "password": "supersecret",
        "secret": "supersecret",
        "nornir_settings": {
            #"credentials": "nautobot_plugin_nornir.plugins.credentials.env_vars.CredentialsEnvVars",
            "runner": {
                "plugin": "threaded",
                "options": {
                    "num_workers": 20,
                },
            },
        },
    },
.....
}

Expected Behavior

If a setting is required like credential fail sanely.

Observed Behavior

Traceback with limited information on how to fix the problem.

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/nautobot/extras/jobs.py", line 1041, in _run_job
    output = job.run(data=data, commit=commit)
  File "/opt/nautobot/.local/lib/python3.9/site-packages/nautobot_golden_config/jobs.py", line 48, in inner
    return method(obj, data, commit)
  File "/opt/nautobot/.local/lib/python3.9/site-packages/nautobot_golden_config/jobs.py", line 103, in run
    config_compliance(self, data)
  File "/opt/nautobot/.local/lib/python3.9/site-packages/nautobot_golden_config/nornir_plays/config_compliance.py", line 142, in config_compliance
    with InitNornir(
  File "/opt/nautobot/.local/lib/python3.9/site-packages/nornir/init_nornir.py", line 72, in InitNornir
    inventory=load_inventory(config),
  File "/opt/nautobot/.local/lib/python3.9/site-packages/nornir/init_nornir.py", line 20, in load_inventory
    inv = inventory_plugin(**config.inventory.options).load()
  File "/opt/nautobot/.local/lib/python3.9/site-packages/nautobot_plugin_nornir/plugins/inventory/nautobot_orm.py", line 89, in __init__
    self.cred_class = import_string(credentials_class)
  File "/usr/local/lib/python3.9/site-packages/django/utils/module_loading.py", line 13, in import_string
    module_path, class_name = dotted_path.rsplit('.', 1)
AttributeError: 'NoneType' object has no attribute 'rsplit'

Enable FQDN Toggle

Add the ability to add fqdn to all devices names.

ORM Inventory Grouping

Allow user/operator defined grouping via the ORM inventory

Add Pyntc as a default

https://github.com/nautobot/nautobot-plugin-nornir/blob/53e89de3012ab942da2a318d5d7055d86689aaa4/nautobot_plugin_nornir/plugins/inventory/nautobot_orm.py#L201

Now that nornir-pyntc is open sourced it'd make sense to add it in as a default option.

Supported Credential Type

Support more credential types.

Better error handling when using `config_context` for secret_access_type.

Environment

Python version: latest
Nautobot version: latest
nautobot-plugin-nornir version: latest

Proposed Functionality

Pre-emptivly check, and allow sane error

    access_type_str = device_obj.get_config_context()["nautobot_plugin_nornir"]["secret_access_type"].upper()
KeyError: 'nautobot_plugin_nornir'

Use Case

More clear guidance to the user.

Admin Guide link in Readme is broken

Admin guide link in Readme leads to 404.

Update Testing

Testing is near non-existent, needs to be updated.

Potential dependency conflict with Nautobot

In pyproject.toml the dev dependencies for this plugin should probably be pinned to "*" to inherit the actual version from Nautobot.

Update CI testing (including Upstream Testing)

Enable Upstream Testing
Update workflows/ci.yml with new standards to not duplicate jobs, etc.

Follow similar pattern as to: nautobot/nautobot-app-chatops#171

CredentialsSettingsVars "There was no username defined, preemptively failed"

Environment

Python version: 3.6.8
Nautobot version: 1.1.2
nautobot-plugin-nornir version: 0.9.4

Steps to Reproduce

Use the CredentialsSettingsVars and defined credentials in the nautobot_config.py
Attempt to backup a config (in my case Juniper)

Expected Behavior

Backup of configuration using credentials defined

Observed Behavior

It appears to select the correct platform but is not using the defined credentials as per the below

Executing dispatcher for cpe1 (juniper_junos)
Found driver nornir_nautobot.plugins.tasks.dispatcher.juniper_junos.NautobotNornirDriver
There was no username defined, preemptively failed

Update readme

Incorrect setup instructions on PyPi

Environment

Python version: 3.8
Nautobot version: 1.0.0
nautobot-plugin-nornir version: 0.9.0

The documentation hosted on PyPi incorrectly refers to the package name as nautobot_nornir for both the plugin and plugins_config sections

Steps to Reproduce

open a web browser
visit pypi page https://pypi.org/project/nautobot-plugin-nornir/
look at the documentation

Expected Behavior

Expected to read "nautobot_plugin_nornir"

Oberved reading "nautobot_nornir"

Job Allowlist

Expose as each nornir task as a job that is allowlisted.

Too Many Clients Error when running a Job

Environment

Python version: 3.7+
Nautobot version: 1.2.x
nautobot-plugin-nornir version: current + develop branch

Steps to Reproduce

Setup golden config plugin
Run against high number of devices.

Currently an intermittent issue that has only been hit a few times without exact replication steps.

Expected Behavior

Job completes as expected.

Observed Behavior

Job succeeds and then every device thereafter fails with too many clients error in the job result.

Initial Theories

the problem is a conflict between uwsgi and celery spawning processes and leaving connections open
basically by setting to serial, we just leave it up to uwsgi to thread it and the connections aren’t left open

This is a symptom not the issue. Needs to be investigated further.

Workaround

Set the runner to serial and disabled the num of workers.

Golden Config but not needing this plugin

Environment

Python version: 3.6
Nautobot version: 1.1.3
nautobot-plugin-nornir version: 0.9.4

Steps to Reproduce

Install Golden Config Plugin (https://github.com/nautobot/nautobot-plugin-golden-config/blob/develop/docs/installation.md)
Only setup required Golden Config Plugin settings within nautobot_config.py

  nautobot_golden_config:
    enable_intended: true

Run nautobot-server migrate

Expected Behavior

That the default settings would be inherited and not crash while attempting to apply the migrations

Observed Behavior

   Traceback (most recent call last):
      File "/appserver/nautobot/bin/nautobot-server", line 8, in <module>
        sys.exit(main())
      File "/appserver/nautobot/lib64/python3.6/site-packages/nautobot/core/cli.py", line 62, in main
        initializer=_configure_settings,  # Called after defaults
      File "/appserver/nautobot/lib64/python3.6/site-packages/nautobot/core/runner/runner.py", line 266, in run_app
        management.execute_from_command_line([runner_name, command] + command_args)
      File "/appserver/nautobot/lib64/python3.6/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
        utility.execute()
      File "/appserver/nautobot/lib64/python3.6/site-packages/django/core/management/__init__.py", line 377, in execute
        django.setup()
      File "/appserver/nautobot/lib64/python3.6/site-packages/django/__init__.py", line 24, in setup
        apps.populate(settings.INSTALLED_APPS)
      File "/appserver/nautobot/lib64/python3.6/site-packages/django/apps/registry.py", line 122, in populate
        app_config.ready()
      File "/appserver/nautobot/lib64/python3.6/site-packages/nautobot/extras/plugins/__init__.py", line 94, in ready
        jobs = import_object(f"{self.__module__}.{self.jobs}")
      File "/appserver/nautobot/lib64/python3.6/site-packages/nautobot/extras/plugins/utils.py", line 44, in import_object
        spec.loader.exec_module(module)
      File "<frozen importlib._bootstrap_external>", line 678, in exec_module
      File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
      File "/appserver/nautobot/lib64/python3.6/site-packages/nautobot_golden_config/jobs.py", line 13, in <module>
        from nautobot_golden_config.nornir_plays.config_intended import config_intended
      File "/appserver/nautobot/lib64/python3.6/site-packages/nautobot_golden_config/nornir_plays/config_intended.py", line 17, in <module>
        from nautobot_plugin_nornir.constants import NORNIR_SETTINGS
      File "/appserver/nautobot/lib64/python3.6/site-packages/nautobot_plugin_nornir/constants.py", line 11, in <module>
        PLUGIN_CFG = settings.PLUGINS_CONFIG["nautobot_plugin_nornir"]
    KeyError: 'nautobot_plugin_nornir'

And then if I set nautobot_plugin_nornir as blank dictionary, I get the following:

    Traceback (most recent call last):
      File "/appserver/nautobot/bin/nautobot-server", line 8, in <module>
        sys.exit(main())
      File "/appserver/nautobot/lib64/python3.6/site-packages/nautobot/core/cli.py", line 62, in main
        initializer=_configure_settings,  # Called after defaults
      File "/appserver/nautobot/lib64/python3.6/site-packages/nautobot/core/runner/runner.py", line 266, in run_app
        management.execute_from_command_line([runner_name, command] + command_args)
      File "/appserver/nautobot/lib64/python3.6/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
        utility.execute()
      File "/appserver/nautobot/lib64/python3.6/site-packages/django/core/management/__init__.py", line 377, in execute
        django.setup()
      File "/appserver/nautobot/lib64/python3.6/site-packages/django/__init__.py", line 24, in setup
        apps.populate(settings.INSTALLED_APPS)
      File "/appserver/nautobot/lib64/python3.6/site-packages/django/apps/registry.py", line 122, in populate
        app_config.ready()
      File "/appserver/nautobot/lib64/python3.6/site-packages/nautobot/extras/plugins/__init__.py", line 94, in ready
        jobs = import_object(f"{self.__module__}.{self.jobs}")
      File "/appserver/nautobot/lib64/python3.6/site-packages/nautobot/extras/plugins/utils.py", line 44, in import_object
        spec.loader.exec_module(module)
      File "<frozen importlib._bootstrap_external>", line 678, in exec_module
      File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
      File "/appserver/nautobot/lib64/python3.6/site-packages/nautobot_golden_config/jobs.py", line 13, in <module>
        from nautobot_golden_config.nornir_plays.config_intended import config_intended
      File "/appserver/nautobot/lib64/python3.6/site-packages/nautobot_golden_config/nornir_plays/config_intended.py", line 17, in <module>
        from nautobot_plugin_nornir.constants import NORNIR_SETTINGS
      File "/appserver/nautobot/lib64/python3.6/site-packages/nautobot_plugin_nornir/constants.py", line 12, in <module>
        NORNIR_SETTINGS = PLUGIN_CFG.get("nornir_settings", _NORNIR_SETTINGS)
    AttributeError: 'NoneType' object has no attribute 'get'`

In this case, we're not running the backup due to environment restrictions so we really don't need this plugin and expect the defaults to work since this doesn't appear to require anything.

I believe the bug fix is to change nautobot_nornir_plugin/constants.py

PLUGIN_CFG = settings.PLUGINS_CONFIG.get("nautobot_plugin_nornir", {})
NORNIR_SETTINGS = PLUGIN_CFG.get("nornir_settings", _NORNIR_SETTINGS)

I'm not entirely sure if this is more or less a bug with Nautobot and how it is setting the plugin settings so we can potentially ask that as well.

Update FAQ

Migrate to Nornir transformers?

Nornir supports transformers for a use case similar to credentials, perhaps this should be the future?

Improve CredentialsSettingsVars documentation

I think it would be a good idea to update the README.md of ntc-nautobot-plugin-nornir to replace the sample config:

PLUGINS_CONFIG = {
    "nautobot_plugin_nornir": {
        # ...
        "dispatcher_mapping": None,
        "username": "ntc",
        "password": "password123",
        "secret": "password123",
    }
}

with

PLUGINS_CONFIG = {
    "nautobot_plugin_nornir": {
        "nornir_settings": {
            "credentials": "nautobot_plugin_nornir.plugins.credentials.settings_vars.CredentialsSettingsVars
            # ...
        },
        # ...
        "dispatcher_mapping": None,
        "username": "ntc",
        "password": "password123",
        "secret": "password123",
    }
}

It would be much more explicit and avoid some mistake

Allow users to provide extra netmiko and Napalm settings

Environment

Python version: 3.7
Nautobot version: 11.1.6
nautobot-plugin-nornir version: 0.9.7

Proposed Functionality

Netmiko and Napalm connection plugins for Nornir both support some additional parameters via the extras section of the connection plugin.
The proposal is to extend the Nornir Inventory to allow uses to specific a list of extras parameters and pass them to Nornir.
See below an example of what the configuration could look like

PLUGINS_CONFIG = {
  "nautobot_plugin_nornir": {
    "napalm_extras": {
         .. 
     },
    "netmiko_extras": {
      "global_delay_factor": 5,
      "banner_timeout":  40, 
      "conn_timeout": 30,
      },
    },
  }

Use Case

Allow users to customize Netmiko and Napalm behavior. One of the primary use case is to adjust some timers and timeout conn_timeout etc ..

Problem : Undefined environment variables

Hi.

Nautobot is able to reach / authenticate my network devices using secrets + env variables.

For some strange reason when i try run a backup job, i get a message stating that the env variables are not available.


"SecretValueNotFoundError: Secret "cisco-user" (provider "EnvironmentVariableSecretsProvider"): Undefined environment variable "NAPALM_USERNAME"! "

here is my nautobot plugin config:

PLUGINS = ["nautobot_plugin_nornir", "nautobot_golden_config", "nautobot_device_lifecycle_mgmt"]

PLUGINS_CONFIG = {
    "nautobot_plugin_nornir": {
        "use_config_context": {"secrets": True, "connection_options": True},
        # Optionally set global connection options.
        "connection_options": {
            "napalm": {
                "extras": {
                    "optional_args": {"global_delay_factor": 1},
                },
            },
            "netmiko": {
                "extras": {
                    "global_delay_factor": 1,
                },
            },
        },
        "nornir_settings": {
            "credentials": "nautobot_plugin_nornir.plugins.credentials.nautobot_secrets.CredentialsNautobotSecrets",
            "runner": {
                "plugin": "threaded",
                "options": {
                    "num_workers": 20,
                },
            },
        },
    },
    "nautobot_golden_config": {
        "per_feature_bar_width": 0.15,
        "per_feature_width": 13,
        "per_feature_height": 4,
        "enable_backup": True,
        "enable_compliance": True,
        "enable_intended": True,
        "enable_sotagg": True,
        "sot_agg_transposer": None,
        "platform_slug_map": None,
        # "get_custom_compliance": "my.custom_compliance.func"
    },

}

Thanks in advance.

nautobot / nautobot-app-nornir Goto Github PK

nautobot-app-nornir's Introduction

Nautobot

Key Use Cases

Build Status

Screenshots

Installation

Application Stack

Plugins and Extensibility

Plugin Screenshots

Golden Config Plugin

ChatOps Plugin

Device Lifecycle Management Plugin

Providing Feedback

Related projects

Notices

nautobot-app-nornir's People

Contributors

Stargazers

Watchers

Forkers

nautobot-app-nornir's Issues

Environment

Proposed Functionality

Use Case

Environment

Proposed Functionality

Use Case

Environment

Proposed Functionality

Use Case

Environment

Proposed Functionality

Use Case

Environment

Expected Behavior

Observed Behavior

Environment

Steps to Reproduce

Expected Behavior

Observed Behavior

Environment

Steps to Reproduce

Expected Behavior

Observed Behavior

Environment

Proposed Functionality

Use Case

Environment

Steps to Reproduce

Expected Behavior

Observed Behavior

Environment

Steps to Reproduce

Expected Behavior

Environment

Steps to Reproduce

Expected Behavior

Observed Behavior

Initial Theories

Workaround

Environment

Steps to Reproduce

Expected Behavior

Observed Behavior

Environment

Proposed Functionality

Use Case

Recommend Projects

Recommend Topics

Recommend Org