nautobot / nautobot-plugin-ssot-arista-cloudvision Goto Github PK

Environment

• Python version: 3.10.6
• Nautobot version: 1.5.16
• nautobot-ssot: 1.2.0
• nautobot-ssot-aristacv: 1.6.0
• CloudVision on-prem: 2022.3.0

Expected Behavior

Sync between CVP and nautobot

Observed Behavior

Error running CloudVision ⟹ Nautobot job

Steps to Reproduce

1. Run sync job

troubleshooting steps

I downgraded nautobot-ssot-aristacv to 1.5.1 and the sync job runs properly with no issues.

Environment

• Python version: 3.9
• Nautobot version: 1.0.3
• aristacv-sync version: 1.0.1
• CVP: 2021.1.1

Expected Behavior

Sync works

Observed Behavior

Sync fails

Steps to Reproduce

1. Create group/containers hierarchy in CVP
2. Put device in the leaf
3. Start sync

Additional details

In CVP 2021.1.1 (at least), when you place devices in nested container, the device inherits multiple Container tags, which results in errors when syncing:

Traceback (most recent call last):
  File "/opt/nautobot/.local/lib/python3.9/site-packages/nautobot_ssot_aristacv/diffsync/fromcv/cloudvision.py", line 39, in load
    self.add(self.cf)
  File "/opt/nautobot/.local/lib/python3.9/site-packages/diffsync/__init__.py", line 657, in add
    raise ObjectAlreadyExists(f"Object {uid} already present")
diffsync.exceptions.ObjectAlreadyExists: Object arista_Container__<device_name> already present

Environment

• Nautobot version: v1.3.4
• aristacv-sync version: 1.4.0

Proposed Functionality

# pylint: disable=invalid-name
"""Nautobot signal handler functions for aristavc_sync."""

from django.apps import apps as global_apps
from nautobot.extras.choices import CustomFieldTypeChoices


def post_migrate_create_custom_fields(apps=global_apps, **kwargs):
    """Callback function for post_migrate() -- create CustomField records."""
    ContentType = apps.get_model("contenttypes", "ContentType")
    Device = apps.get_model("dcim", "Device")
    CustomField = apps.get_model("extras", "CustomField")

    for device_cf_dict in [
        {
            "name": "arista_eostrain",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "EOS Train",
        },
        {
            "name": "arista_eos",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "EOS Version",
        },
        {
            "name": "arista_ztp",
            "type": CustomFieldTypeChoices.TYPE_BOOLEAN,
            "label": "ztp",
        },
        {
            "name": "arista_pimbidir",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "pimbidir",
        },
        {
            "name": "arista_pim",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "pim",
        },
        {
            "name": "arista_bgp",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "bgp",
        },
        {
            "name": "arista_mpls",
            "type": CustomFieldTypeChoices.TYPE_BOOLEAN,
            "label": "mpls",
        },
        {
            "name": "arista_systype",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "systype",
        },
        {
            "name": "arista_mlag",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "mlag",
        },
        {
            "name": "arista_tapagg",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "TAP Aggregation",
        },
        {
            "name": "arista_sflow",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "sFlow",
        },
        {
            "name": "arista_terminattr",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "TerminAttr Version",
        },
        {
            "name": "arista_topology_network_type",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "Topology Network Type",
        },
        {
            "name": "arista_topology_type",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "Topology Type",
        },
        {
            "name": "arista_topology_pod",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "Topology pod",
        },
    ]:
        print("-------------------------------------------")
        print(device_cf_dict)
        field, created = CustomField.objects.get_or_create(
            name=device_cf_dict["name"],
            defaults=device_cf_dict,
            slug=device_cf_dict["name"],
        )
        print(field, created)
        field.content_types.set([ContentType.objects.get_for_model(Device)])


def post_migrate_create_manufacturer(apps=global_apps, **kwargs):
    """Callback function for post_migrate() -- create Arista Manufacturer."""
    Manufacturer = apps.get_model("dcim", "Manufacturer")
    Manufacturer.objects.get_or_create(name="Arista", slug="arista")


def post_migrate_create_platform(apps=global_apps, **kwargs):
    """Callback function for post_migrate() -- create Arista Platform."""
    Platform = apps.get_model("dcim", "Platform")
    Manufacturer = apps.get_model("dcim", "Manufacturer")
    Platform.objects.get_or_create(
        name="arista.eos.eos",
        slug="arista_eos",
        napalm_driver="eos",
        manufacturer=Manufacturer.objects.get(slug="arista"),
    )

Use Case

Switch A - Switch B (pod1)
Switch C - Switch D (pod2)
Switch E - Switch F (pod3)
I want to have two free interfaces of my pod1, so to know which switches is my pod 1 , I search for pod1 in the field arista_topology_rack.

The pod1 information is provide from cloudvision configuration.

Environment

• Python version: <3.8>
• Nautobot version: <1.4.5>
• aristacv-sync version: <1.3.0>

Data imported correctly into nautobot from CVP.

Observed Behavior

import failed with teh following error :-

An exception occurred: KeyError: 'enabledState'

Traceback (most recent call last):
File "/opt/data/nautobot/lib64/python3.8/site-packages/nautobot_ssot/jobs/base.py", line 332, in run
self.sync_data()
File "/opt/data/nautobot/lib64/python3.8/site-packages/nautobot_ssot_aristacv/jobs.py", line 135, in sync_data
cv.load()
File "/opt/data/nautobot/lib64/python3.8/site-packages/nautobot_ssot_aristacv/diffsync/adapters/cloudvision.py", line 126, in load
self.load_devices()
File "/opt/data/nautobot/lib64/python3.8/site-packages/nautobot_ssot_aristacv/diffsync/adapters/cloudvision.py", line 45, in load_devices
self.load_interfaces(device=new_device)
File "/opt/data/nautobot/lib64/python3.8/site-packages/nautobot_ssot_aristacv/diffsync/adapters/cloudvision.py", line 58, in load_interfaces
port_info = cloudvision.get_interfaces_chassis(client=self.conn, dId=device.serial)
File "/opt/data/nautobot/lib64/python3.8/site-packages/nautobot_ssot_aristacv/utils/cloudvision.py", line 512, in get_interfaces_chassis
"enabled": bool(results["enabledState"]["Name"] == "enabled"),
KeyError: 'enabledState'

Environment

• Nautobot version: v1.3.4
• aristacv-sync version: 1.4.0

Proposed Functionality

# pylint: disable=invalid-name
"""Nautobot signal handler functions for aristavc_sync."""

from django.apps import apps as global_apps
from nautobot.extras.choices import CustomFieldTypeChoices


def post_migrate_create_custom_fields(apps=global_apps, **kwargs):
    """Callback function for post_migrate() -- create CustomField records."""
    ContentType = apps.get_model("contenttypes", "ContentType")
    Device = apps.get_model("dcim", "Device")
    CustomField = apps.get_model("extras", "CustomField")

    for device_cf_dict in [
        {
            "name": "arista_eostrain",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "EOS Train",
        },
        {
            "name": "arista_eos",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "EOS Version",
        },
        {
            "name": "arista_ztp",
            "type": CustomFieldTypeChoices.TYPE_BOOLEAN,
            "label": "ztp",
        },
        {
            "name": "arista_pimbidir",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "pimbidir",
        },
        {
            "name": "arista_pim",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "pim",
        },
        {
            "name": "arista_bgp",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "bgp",
        },
        {
            "name": "arista_mpls",
            "type": CustomFieldTypeChoices.TYPE_BOOLEAN,
            "label": "mpls",
        },
        {
            "name": "arista_systype",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "systype",
        },
        {
            "name": "arista_mlag",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "mlag",
        },
        {
            "name": "arista_tapagg",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "TAP Aggregation",
        },
        {
            "name": "arista_sflow",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "sFlow",
        },
        {
            "name": "arista_terminattr",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "TerminAttr Version",
        },
        {
            "name": "arista_topology_network_type",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "Topology Network Type",
        },
        {
            "name": "arista_topology_type",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "Topology Type",
        },
        {
            "name": "arista_topology_pod",
            "type": CustomFieldTypeChoices.TYPE_TEXT,
            "label": "Topology pod",
        },
    ]:
        print("-------------------------------------------")
        print(device_cf_dict)
        field, created = CustomField.objects.get_or_create(
            name=device_cf_dict["name"],
            defaults=device_cf_dict,
            slug=device_cf_dict["name"],
        )
        print(field, created)
        field.content_types.set([ContentType.objects.get_for_model(Device)])


def post_migrate_create_manufacturer(apps=global_apps, **kwargs):
    """Callback function for post_migrate() -- create Arista Manufacturer."""
    Manufacturer = apps.get_model("dcim", "Manufacturer")
    Manufacturer.objects.get_or_create(name="Arista", slug="arista")


def post_migrate_create_platform(apps=global_apps, **kwargs):
    """Callback function for post_migrate() -- create Arista Platform."""
    Platform = apps.get_model("dcim", "Platform")
    Manufacturer = apps.get_model("dcim", "Manufacturer")
    Platform.objects.get_or_create(
        name="arista.eos.eos",
        slug="arista_eos",
        napalm_driver="eos",
        manufacturer=Manufacturer.objects.get(slug="arista"),
    )

Use Case

Switch A - Switch B (pod1)
Switch C - Switch D (pod2)
Switch E - Switch F (pod3)
I want to have two free interfaces of my pod1, so to know which switches is my pod 1 , I search for pod1 in the field arista_topology_rack.

The pod1 information is provide from cloudvision configuration.

I think that the issue is here:

Environment

• Nautobot version: 1.2.11
• aristacv-sync version: 1.0.5

Proposed Functionality

At the moment there are no API endpoints for this plugin. This means that all sync jobs have to be executed within nautobot. This also prevents the user from executing a sync operation outside of nautobot. I think it would be beneficial to add API endpoints to run sync jobs in either direction.

Use Case

When working with CVP, there can be a workflow to add or remove devices. Once this is complete, a workflow could then be initiated to call the sync from cloudvision to nautbot job. This would then keep nautobot and CVP in sync in a fairly close window. Allowing this option in the form of an endpoint would allow the user to initiate this from whatever workflow currently exists in their environment.

Environment

• Python version: 3.9
• Nautobot version: 1.4.7
• aristacv-sync version: 0.7.4

Expected Behavior

Expect to see the status for an interface to be Active if the Operational status is up.

Observed Behavior

The operational and link status keys aren't always in the same update notification as the interface ID, MTU, and other attributes but are seen in subsequent notifications. As they're not seen in the same notification they're assumed to be down. This doesn't match what is seen in CloudVision.

Steps to Reproduce

1. Setup Nautobot and CVP SSoT plugin.
2. Run import from CloudVision to Nautobot.
3. Review the interface Status for Devices and compare to CVP. Notice how a large number are Maintenance but enabled.

When I try to run a sync from the CloudVision data source using CVaaS, I get the following traceback:

An exception occurred: ValidationError: {'all': ["Unknown field name 'arista_model' in custom field data."]}

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/nautobot/extras/jobs.py", line 832, in _run_job
output = job.run(data=data, commit=commit)
File "/usr/local/lib/python3.6/site-packages/nautobot_ssot/jobs/base.py", line 202, in run
self.sync_data()
File "/source/nautobot_ssot_aristacv/jobs.py", line 47, in sync_data
cv.sync_to(nb)
File "/usr/local/lib/python3.6/site-packages/diffsync/init.py", line 496, in sync_to
target.sync_from(self, diff_class=diff_class, flags=flags, callback=callback)
File "/usr/local/lib/python3.6/site-packages/diffsync/init.py", line 476, in sync_from
result = syncer.perform_sync()
File "/usr/local/lib/python3.6/site-packages/diffsync/helpers.py", line 313, in perform_sync
changed |= self.sync_diff_element(element)
File "/usr/local/lib/python3.6/site-packages/diffsync/helpers.py", line 368, in sync_diff_element
changed |= self.sync_diff_element(child, parent_model=model)
File "/usr/local/lib/python3.6/site-packages/diffsync/helpers.py", line 345, in sync_diff_element
changed, modified_model = self.sync_model(model, ids, attrs)
File "/usr/local/lib/python3.6/site-packages/diffsync/helpers.py", line 401, in sync_model
model = model.delete()
File "/source/nautobot_ssot_aristacv/diffsync/fromcv/models.py", line 90, in delete
device.validated_save()
File "/usr/local/lib/python3.6/site-packages/nautobot/core/models/init.py", line 51, in validated_save
self.full_clean()
File "/usr/local/lib/python3.6/site-packages/django/db/models/base.py", line 1231, in full_clean
raise ValidationError(errors)
django.core.exceptions.ValidationError: {'all': ["Unknown field name 'arista_model' in custom field data."]}

Environment

• Python version: 3.9
• Nautobot version: 1.5.10
• aristacv-sync version: 1.5.1

Expected Behavior

A sync of plugin SSOT between Cloudvision and Nautobot in our dev environnement lasts 29 minutes with : {'create': 3404, 'update': 0, 'delete': 0, 'no-change': 0, 'skip': 0} ( for 20 devices)

So the behavior expected is less then 1hours for a normal sync

Observed Behavior

A sync of plugin SSOT between Cloudvision and Nautobot in our prod environnement lasts more than 19 hours and it still running with : {'create': 39685, 'update': 22, 'delete': 3, 'no-change': 2774, 'skip': 1}

The job remains stuck at this stage : Syncing from CloudvisionAdapter to NautobotAdapter...
And it never finish .

Could you check why this sync is very slow ?

Steps to Reproduce

1. Sync Cloudvision to Nautobot ( with 145 devices in CvaaS)
2. Wait and you wil see

The README still references the old module name and needs to be updated. Since the README is generic, we'll hold this issue until we're ready to document.

This is a reminder to update the README prior to release.

Environment

• Python version: 3.7
• Nautobot version: 1.2.5
• aristacv-sync version: 1.0.5

Fix the pyproject.toml to allow newer versions of nautobot-ssot.

Expected Behavior

I should be able to update nautobot-ssot to the latest.

Observed Behavior

nautobot-ssot is pinned at version 1.0.1 by this plugin.

Environment

• Python version: 3.8.13
• Nautobot version: v1.3.2
• aristacv-sync version: 1.0.5

I have changed the cvp default port to 443 via customizing the variables in nautobot config.py. This is my configuration.

PLUGINS_CONFIG = {
"nautobot_ssot": {
"hide_example_jobs": True, # defaults to False if unspecified
},
"nautobot_ssot_aristacv": {
"cvp_host": "10.23.84.200",
"cvp_port": "443",
"cvp_user": "gcastaldo",
"cvp_password": "xxxxxx",
"insecure": "True"
}
}

I expect that the socket will be opened on port 443.

Observed Behavior

I experienced error on socket opens.

Traceback (most recent call last):
File "/opt/nautobot/.local/lib/python3.8/site-packages/nautobot_ssot/jobs/base.py", line 207, in run
self.sync_data()
File "/opt/nautobot/.local/lib/python3.8/site-packages/nautobot_ssot_aristacv/jobs.py", line 125, in sync_data
cvutils.connect()
File "/opt/nautobot/.local/lib/python3.8/site-packages/nautobot_ssot_aristacv/diffsync/cvutils.py", line 41, in connect
cert = bytes(ssl.get_server_certificate((cvp_host, 8443)), "utf-8")
File "/usr/local/lib/python3.8/ssl.py", line 1483, in get_server_certificate
with create_connection(addr) as sock:
File "/usr/local/lib/python3.8/socket.py", line 808, in create_connection
raise err
File "/usr/local/lib/python3.8/socket.py", line 796, in create_connection
sock.connect(sa)
OSError: [Errno 113] No route to host

but, via tcpdump, i checked that plugin tries to open the socket on port 8443 .

root@VMW-DEVNET-AWX17:~/nautobot-docker-compose# tcpdump -i ens32 host 10.23.84.200
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens32, link-type EN10MB (Ethernet), capture size 262144 bytes
11:50:22.344580 IP VMW-DEVNET-AWX17.maticmind.it.56764 > 10.23.84.200.8443: Flags [S], seq 1288640698, win 64240, options [mss 1460,sackOK,TS val 3279922292 ecr 0,nop,wscale 7], length 0
11:50:22.355716 IP 10.23.84.200 > VMW-DEVNET-AWX17.maticmind.it: ICMP host 10.23.84.200 unreachable - admin prohibited, length 76

Steps to Reproduce

1. Install nautobot (in my case i installed natubot in docker)
2. Install and configure nautobot_ssot and nautobot_ssot_aristacv
3. Change the nautobot_config.py customizing the field for the nautobot_ssot and nautobot_ssot_aristacv
4. Run the sync in nautobot ssot dashboard.

Environment

• Nautobot version: 1.0.1
• aristacv-sync version: 1.0.2

Proposed Functionality

When importing devices into Nautobot, allow for a management IP address to be synced as well.

Use Case

This would be helpful in syncing more information about devices into Nautobot which means Nautobot could be the Source of Truth for more data.

Environment

• Python version: N/A
• Nautobot version: N/A
• aristacv-sync version: 1.0.6

Expected Behavior

Links from PyPI to home page and repository to bring me to correct page.

Observed Behavior

You get a 404 Page Not Found error.

Steps to Reproduce

1. Look at plug-in on PyPI.
2. Attempt to go to homepage or repo.

Environment

• Python version: 3.7
• Nautobot version: 1.4.5
• aristacv-sync version: 1.3.0

Expected Behavior

I expect a single platform to be created and assigned to all Devices. The single Platform should be for Arista.

Observed Behavior

A platform for each device type is created.

Steps to Reproduce

1. Stand-up Nautobot with CVP SSoT plugin installed.
2. Run import of devices.
3. Look at the Platforms list.

Environment

• Python version: 3.8
• Nautobot version: 1.4.7
• aristacv-sync version: 1.4.0

Expected Behavior

I'm expecting to get custom fields from migration that include the prefix "arista_" in the slug.

Observed Behavior

Custom fields are created, but without the prefix arista_.
Got a KeyError 'arista_eos' on sync from cloudvision to nautobot
Got a warning icon next to the slug column items saying : 'Name "all fields" does not match slug'

Steps to Reproduce

From a fresh db.

1. nautobot-server -c nautobot_config.py migrate
2. Go to custom fields sub section from the Extensibility section

Environment

• Nautobot version: 1.0.1
• aristacv-sync version: 1.0.2

Proposed Functionality

Rather than having a blanket default site and/or device role, provide some method that allows customizing the site and/or device roles applied to devices when imported.

Use Case

As an end user using the current state of the plugin, changing device roles and device sites must be down manually after importing data in Nautobot. Allowing for customization of device role and sites would streamline the import process.

Environment

• Python version: 3.8.12
• Nautobot version: 1.4.3
• aristacv-sync version: 1.2.0

When syncing data from CVP to Nautobot, plugin fails when "Loading data from Nautobot".
The plugin apparently try to validate all custom fields, not just "it's own" or the ones created/handled by the plugin.
The plugin should only care about it's own fields.

The example happens because the fieldvalidation is done for all custom fields.

An exception occurred: ValidationError: 1 validation error for CustomField value str type expected (type=type_error.str)

Traceback (most recent call last):
  File "/opt/nautobot/lib64/python3.8/site-packages/nautobot_ssot/jobs/base.py", line 327, in run
    self.sync_data()
  File "/opt/nautobot/lib64/python3.8/site-packages/nautobot_ssot_aristacv/jobs.py", line 131, in sync_data
    nb.load()
  File "/opt/nautobot/lib64/python3.8/site-packages/nautobot_ssot_aristacv/diffsync/fromcv/nautobot.py", line 34, in load
    self.cf = CustomField(name=cf_name, value=cf_value, device_name=dev.name)
  File "pydantic/main.py", line 331, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for CustomField
value
  str type expected (type=type_error.str)__

Environment

• Nautobot version: develop-latest
• aristacv-sync version: 1.0.1

Proposed Functionality

Create a cloudvision_imported tag tied to devices imported into Nautobot from CV.

Use Case

This would allow to see which devices have been imported when you override the default import settings.

Environment

• Nautobot version: 1.4.1
• aristacv-sync version: 1.2.1

Proposed Functionality

It'd be nice to have the Software Version of a Device be noted in the Device Lifecycle plugin if it's also imported.

Use Case

Instead of having EOS Version noted as a Custom Field it'd be much better to have those versions created in the DLC plugin and a Relationship created. This enables far more functionality with this information, such as defining Validated Software or CVEs.

Environment

• Python version: 3.9.16
• Nautobot version: v1.5.9
• aristacv-sync version: 1.5.0

Expected Behavior

The interfaces in switchport trunk mode are recognized and configured in mode "TAGGED"

Observed Behavior

The interfaces in switchport trunk mode aren't recognized and configured in mode "ACCESS"

Hi, I think that it isn't a bug of the plugin because the CVaaS don't report the correct mode of the switchport interfaces.

I have a propose, if the CVaaS output is wrong is better to don't configure it in Nautobot.

Steps to Reproduce

1. To reproduce the question do you need to have a interface configured in switch trunk .
2. Syncing CVaaS or CVP to nautobot
3. Verify the configured interface in Nautobot.

As a Nautobot user, I want to sync the tags assigned to interfaces in Nautobot to CloudVision as interface tags.

The deletion of devices when syncing from CloudVision to Nautobot should be configurable by the end user and default to off to prevent inadvertently deleting non-CloudVision managed devices.

Wouldn't it be better to apply the arista as a filter, that way you don't have to load all the devices?

Environment

• Nautobot version: 1.5.7
• aristacv-sync version: 1.5.0

We received a news from arista about the collection : arista.tag.v1
It will be obsolete soon.

Proposed Functionality

Could you update this part of code : https://github.com/nautobot/nautobot-plugin-ssot-arista-cloudvision/blob/v1.5.0/nautobot_ssot_aristacv/diffsync/adapters/cloudvision.py to use arista.tag.v2 ?

https://aristanetworks.github.io/cloudvision-apis/models/tag.v2/

Thanks

Environment

• Nautobot version: 1.5.1
• aristacv-sync version: 1.4.0

Proposed Functionality

As Nautobot supports documenting console and power ports it would be useful to pull this information from CloudVision if possible.

Use Case

Being able to track connections for console and power is useful for automation and inventory purposes.

Environment

• Python version: 3.9.13
• Nautobot version: 1.4.8 -> 1.5.7
• aristacv-sync version: 1.5.0

Expected Behavior

When upgrading the Docker image of Nautobot from 1.4.8 to 1.5.7, the post_migrate_create_custom_fields command should passed successfully. The command is run automatically by the container.

Observed Behavior

Instead, an exception occurred complaining about a duplicate key.

During handling of the above exception, another exception occurred:

 

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "extras_customfield_name_key"
DETAIL:  Key (name)=(arista_eostrain) already exists.

 


The above exception was the direct cause of the following exception:

 

Traceback (most recent call last):
  File "/usr/local/bin/nautobot-server", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/nautobot/core/cli.py", line 54, in main
    run_app(
  File "/usr/local/lib/python3.9/site-packages/nautobot/core/runner/runner.py", line 263, in run_app
    management.execute_from_command_line([runner_name, command] + command_args)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.9/site-packages/nautobot/core/management/commands/post_upgrade.py", line 78, in handle
    call_command(
  File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 181, in call_command
    return command.execute(*args, **defaults)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 89, in wrapped
    res = handle_func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/commands/migrate.py", line 268, in handle
    emit_post_migrate_signal(
  File "/usr/local/lib/python3.9/site-packages/django/core/management/sql.py", line 42, in emit_post_migrate_signal
    models.signals.post_migrate.send(
  File "/usr/local/lib/python3.9/site-packages/django/dispatch/dispatcher.py", line 180, in send
    return [
  File "/usr/local/lib/python3.9/site-packages/django/dispatch/dispatcher.py", line 181, in <listcomp>
    (receiver, receiver(signal=self, sender=sender, **named))
  File "/usr/local/lib/python3.9/site-packages/nautobot_ssot_aristacv/signals.py", line 83, in post_migrate_create_custom_fields
    field, _ = CustomField.objects.get_or_create(
  File "/usr/local/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 588, in get_or_create
    return self.create(**params), True
  File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 453, in create

Steps to Reproduce

1. Install Nautobot 1.4.8
2. Install the nautobot_ssot_aristacv plugin
3. Upgrade Nautobot to 1.5.7

Environment

• Python version: 3.8.13
• Nautobot version: 1.4.8
• aristacv-sync version: 1.4.0

Expected Behavior

Syncing from cloudvision as a service to nautobot I expect the databases to be aligned.

Observed Behavior

I receive the following traceback:

Traceback (most recent call last):
File "/opt/nautobot/.local/lib/python3.8/site-packages/nautobot_ssot/jobs/base.py", line 332, in run
self.sync_data()
File "/opt/nautobot/.local/lib/python3.8/site-packages/nautobot_ssot/jobs/base.py", line 137, in sync_data
self.load_source_adapter()
File "/opt/nautobot/.local/lib/python3.8/site-packages/nautobot_ssot_aristacv/jobs.py", line 117, in load_source_adapter
self.source_adapter.load()
File "/opt/nautobot/.local/lib/python3.8/site-packages/nautobot_ssot_aristacv/diffsync/adapters/cloudvision.py", line 190, in load
self.load_devices()
File "/opt/nautobot/.local/lib/python3.8/site-packages/nautobot_ssot_aristacv/diffsync/adapters/cloudvision.py", line 53, in load_devices
self.load_interfaces(device=new_device)
File "/opt/nautobot/.local/lib/python3.8/site-packages/nautobot_ssot_aristacv/diffsync/adapters/cloudvision.py", line 67, in load_interfaces
port_info = cloudvision.get_interfaces_chassis(client=self.conn, dId=device.serial)
File "/opt/nautobot/.local/lib/python3.8/site-packages/nautobot_ssot_aristacv/utils/cloudvision.py", line 516, in get_interfaces_chassis
"mtu": results["mtu"],
KeyError: 'mtu'

Steps to Reproduce

1. Create the image for the docker
2. Configure the parameters for nautobot sot plugin
3. Active the dockers
4. Run the job from Cloudvision to Nautobot sync

Environment

• Nautobot version: 1.3.10
• aristacv-sync version: 1.0.6

Proposed Functionality

Currently this plugin only imports devices that have an Active streaming status. I'd like the ability to import all devices connected to a CVP instance regardless of the current status. We can have it as a plugin setting whether only active devices are imported or not.

Use Case

Perhaps I have a lab instance that has a large number of devices connected to it that I wish to import but I don't want to bring them all online simply to import them in. Could also be useful for importing devices that are currently having issues with streaming that you wish to import anyways.

Python version: 3.10 Nautobot version: 1.5.7 nautobot_ssot_aristacv 1.4.0 ssot : 1.2.0
Steps to Reproduce
1.Plugins, dashboard, single source of truth
2.Cloudvision --> Nautobot
3. Debug and Dry run, Run job now

Pull data from CVP into Nautobot, or at least show me what its going to pull in.

After several logs entries showing interface details for each device-
Loading data from Nautobot
Then a failure - An exception occurred: IndexError: list index out of range

Traceback (most recent call last):
File "/opt/nautobot/lib/python3.10/site-packages/nautobot_ssot/jobs/base.py", line 332, in run
self.sync_data()
File "/opt/nautobot/lib/python3.10/site-packages/nautobot_ssot/jobs/base.py", line 146, in sync_data
self.load_target_adapter()
File "/opt/nautobot/lib/python3.10/site-packages/nautobot_ssot_aristacv/jobs.py", line 123, in load_target_adapter
self.target_adapter.load()
File "/opt/nautobot/lib/python3.10/site-packages/nautobot_ssot_aristacv/diffsync/adapters/nautobot.py", line 41, in load
version=nautobot.get_device_version(dev),
File "/opt/nautobot/lib/python3.10/site-packages/nautobot_ssot_aristacv/utils/nautobot.py", line 107, in get_device_version
version = relations["destination"][software_relation][0].source.version
File "/opt/nautobot/lib/python3.10/site-packages/django/db/models/query.py", line 318, in getitem
return qs._result_cache[0]
IndexError: list index out of range

Environment

• Python version: 3.10.8
• Nautobot version: 1.4.7
• aristacv-sync version: 1.4.0

Sync completed

Getting IndexError
Dry run is successful

Environment

• Python version: 3.10.6
• Nautobot version: 1.4.7
• aristacv-sync version: 1.4.0
• aristacvp version: 2022.3

Sync info from CVP to Nautobot

Failing with below trace
Traceback (most recent call last):
File "/opt/nautobot/lib/python3.10/site-packages/nautobot_ssot/jobs/base.py", line 332, in run
self.sync_data()
File "/opt/nautobot/lib/python3.10/site-packages/nautobot_ssot/jobs/base.py", line 137, in sync_data
self.load_source_adapter()
File "/opt/nautobot/lib/python3.10/site-packages/nautobot_ssot_aristacv/jobs.py", line 117, in load_source_adapter
self.source_adapter.load()
File "/opt/nautobot/lib/python3.10/site-packages/nautobot_ssot_aristacv/diffsync/adapters/cloudvision.py", line 190, in load
self.load_devices()
File "/opt/nautobot/lib/python3.10/site-packages/nautobot_ssot_aristacv/diffsync/adapters/cloudvision.py", line 53, in load_devices
self.load_interfaces(device=new_device)
File "/opt/nautobot/lib/python3.10/site-packages/nautobot_ssot_aristacv/diffsync/adapters/cloudvision.py", line 62, in load_interfaces
chassis_type = cloudvision.get_device_type(client=self.conn, dId=device.serial)
File "/opt/nautobot/lib/python3.10/site-packages/nautobot_ssot_aristacv/utils/cloudvision.py", line 472, in get_device_type
if query["fixedSystem"] is None:
KeyError: 'fixedSystem'

Environment

• Python version: 3.7
• Nautobot version: 1.3.10
• aristacv-sync version: 1.0.6

Expected Behavior

Able to connect to on-prem instance using a token.

Observed Behavior

Using a token is only supported with the CVaaS Cloud instance.

Steps to Reproduce

1. Setup Nautobot and CloudVision SSoT.
2. Setup CV plugin with on-prem host and token.
3. Attempt to run sync.

Environment

• Python version: 3.9
• Nautobot version: 1.4.3
• aristacv-sync version: 1.2.1

Expected Behavior

Expect to be able to use the method to get a device's ID by passing the hostname.

Observed Behavior

Running the method throws a TypeError:

In [43]: req = services.DeviceStreamRequest(patrial_eq_filter=[models.Device(hostname="rtr1")])
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In [43], line 1
----> 1 req = services.DeviceStreamRequest(patrial_eq_filter=[models.Device(hostname="rtr1")])

TypeError: Parameter to MergeFrom() must be instance of same class: expected google.protobuf.StringValue got str.

In [44]: cvp.get_device_id(device_name="rtr1")
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In [44], line 1
----> 1 cvp.get_device_id(device_name="rtr1")

File /usr/local/lib/python3.9/site-packages/nautobot_ssot_aristacv/utils/cloudvision.py:119, in CloudvisionApi.get_device_id(self, device_name)
    116 """Get device_id for device_name from CloudVision inventory."""
    117 device_stub = services.DeviceServiceStub(self.comm_channel)
    118 req = services.DeviceStreamRequest(
--> 119     partial_eq_filter=[models.Device(hostname=device_name, streaming_status=models.STREAMING_STATUS_ACTIVE)]
    120 )
    121 resp = device_stub.GetOne(req)
    122 return resp.value.key.device_id.value

TypeError: Parameter to MergeFrom() must be instance of same class: expected google.protobuf.StringValue got str.

Steps to Reproduce

1. Create connection to CVP.
2. Attempt to execute the get_device_id method by passing a hostname.
3. Look at results.

Environment

• Nautobot version: 1.4.7
• aristacv-sync version: 1.4.0

Proposed Functionality

It'd be helpful to have representation of the CloudVision instance and associated data like software version in Nautobot. This could also include the Relationship between that instance and the connected Devices.

Use Case

In order to know which instance controls a particular Device it'd be helpful to document that Relationship and also for tracking versions in use.

Environment

• Nautobot version: 1.1.2
• aristacv-sync version: 1.0.1

Proposed Functionality

Add the ability to customize the CVAAS url.

Use Case

Not all instances of CVAAS live at www.arista.io. Currently, that URL is hardcoded.

Environment

• Nautobot version: 1.0.1
• aristacv-sync version: 1.0.2

Proposed Functionality

Sync device interfaces from CloudVision to Nautobot when syncing devices.

Use Case

Syncing of interfaces from CloudVision to Nautobot would allow Nautobot to be the SSoT of more info.

Environment

• Python version: 3.6.8
• Nautobot version: 1.1.3
• aristacv-sync version: 1.0.2

Sync from cloudvision to Nautobot it looks like the configuration file dosen't match my the boolean True statment in the configuration file of nautbot

Steps to Reproduce

1. install nautbot standard
2. install nautobot_ssot_aristacv with pip3 install nautobot_ssot_aristacv
3. Update he configuration file with this configuration

PLUGINS = ["nautobot_ssot", "nautobot_ssot_aristacv"]


PLUGINS_CONFIG = {
#   "nautobot_ssot" : {
#     ADD YOUR SETTINGS HERE
#   }
   "nautobot_ssot_aristacv": {
#     "cvaas_token": "",
     "cvp_host": "X",
     "cvp_user": "nautobot",
     "cvp_password": "X",
     "insecure": "True",
#     "from_cloudvision_default_site": "",
#     "from_cloudvision_default_device_role": "",
#     "from_cloudvision_default_device_role_color": "",
#     "from_cloudvision_default_device_status": "",
#     "from_cloudvision_default_device_status_color": "",
#     "delete_devices_on_sync": "",
     "apply_import_tag": "True"
   }
 }

Environment

• Python version: 3.8
• Nautobot version: 1.4.1
• aristacv-sync version: 1.2.1

Expected Behavior

All of the values for the Configuration section for the CloudVisionDataSource and Target should should be shown.

Observed Behavior

The values for Verify and delete devices on sync are both blank. I believe this is due to them being booleans and we need to stringify the values for display.

Steps to Reproduce

1. Setup Nautobot with CVP plugin.
2. Setup plugin configuration settings.
3. Look in the Configuration section for the DataSource/Target.

Environment

• Python version: 3.9
• Nautobot version: 1.3.8
• aristacv-sync version: 1.0.6

Expected Behavior

Nautobot starts without issue and the plugin is available for use.

Observed Behavior

Stacktrace is caused by a missing module:

Traceback (most recent call last):
  File "/usr/local/bin/nautobot-server", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/nautobot/core/cli.py", line 54, in main
    run_app(
  File "/usr/local/lib/python3.9/site-packages/nautobot/core/runner/runner.py", line 266, in run_app
    management.execute_from_command_line([runner_name, command] + command_args)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 395, in execute
    django.setup()
  File "/usr/local/lib/python3.9/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/local/lib/python3.9/site-packages/django/apps/registry.py", line 122, in populate
    app_config.ready()
  File "/usr/local/lib/python3.9/site-packages/nautobot_ssot_aristacv/__init__.py", line 34, in ready
    super().ready()
  File "/usr/local/lib/python3.9/site-packages/nautobot/extras/plugins/__init__.py", line 143, in ready
    jobs = import_object(f"{self.__module__}.{self.jobs}")
  File "/usr/local/lib/python3.9/site-packages/nautobot/extras/plugins/utils.py", line 45, in import_object
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/usr/local/lib/python3.9/site-packages/nautobot_ssot_aristacv/jobs.py", line 15, in <module>
    from nautobot_ssot_aristacv.diffsync.tocv.cloudvision import CloudVision
  File "/usr/local/lib/python3.9/site-packages/nautobot_ssot_aristacv/diffsync/tocv/cloudvision.py", line 4, in <module>
    import nautobot_ssot_aristacv.diffsync.cvutils as cvutils
  File "/usr/local/lib/python3.9/site-packages/nautobot_ssot_aristacv/diffsync/cvutils.py", line 6, in <module>
    import arista.inventory.v1 as inv
  File "/usr/local/lib/python3.9/site-packages/arista/inventory/v1/__init__.py", line 3, in <module>
    from arista.inventory.v1 import inventory_pb2 as models
  File "/usr/local/lib/python3.9/site-packages/arista/inventory/v1/inventory_pb2.py", line 17, in <module>
    from fmp import extensions_pb2 as fmp_dot_extensions__pb2
  File "/usr/local/lib/python3.9/site-packages/fmp/__init__.py", line 3, in <module>
    import wrappers_pb2 as wrappers
ModuleNotFoundError: No module named 'wrappers_pb2'

Steps to Reproduce

1. Build Nautobot
2. Add Arista CV SSoT plugin to environment and enable in PLUGINS.
3. Attempt to start Nautobot.

Environment

• aristacv-sync version: 1.0.1

Hello,

the documentation (README) mentions delete_devices_on_sync_cv_source while the actual variable seems to be delete_devices_on_sync

Also, the setting apply_import_tag is not included in the example of PLUGINS_CONFIG (https://github.com/nautobot/nautobot-plugin-ssot-arista-cloudvision#installation).

Best,

Environment

• Python version: 3.10
• Nautobot version: 1.5.4
• aristacv-sync version: 1.5.0

Expected Behavior

Upgrade from 1.4.0 to 1.5.0 without issues.

Observed Behavior

Getting an error after upgrading to latest and running nautobot-server post_upgrade

Traceback (most recent call last):
  File "/opt/nautobot/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "extras_customfield_name_key"
DETAIL:  Key (name)=(arista_eostrain) already exists.

Steps to Reproduce

1. pip install nautobot-ssot-aristacv --upgrade
2. nautobot-server post_upgrade

More details

I was able to workaround the error by deleting all CustomFields from the Nautobot instance and try again. After that the error is gone and the upgrades succeeds.

Another issue that came with the upgrade to 1.5.0 is that enabling the create controller feature causes some error on the imported CloudVision device. After the initial sync, the CloudVision controller is created with empty CustomFields values. After trying subsequent syncs, there is an error:

Traceback (most recent call last):
  File "/opt/nautobot/lib/python3.10/site-packages/nautobot_ssot/jobs/base.py", line 332, in run
    self.sync_data()
  File "/opt/nautobot/lib/python3.10/site-packages/nautobot_ssot/jobs/base.py", line 146, in sync_data
    self.load_target_adapter()
  File "/opt/nautobot/lib/python3.10/site-packages/nautobot_ssot_aristacv/jobs.py", line 115, in load_target_adapter
    self.target_adapter.load()
  File "/opt/nautobot/lib/python3.10/site-packages/nautobot_ssot_aristacv/diffsync/adapters/nautobot.py", line 140, in load
    self.load_devices()
  File "/opt/nautobot/lib/python3.10/site-packages/nautobot_ssot_aristacv/diffsync/adapters/nautobot.py", line 45, in load_devices
    version=nautobot.get_device_version(dev),
  File "/opt/nautobot/lib/python3.10/site-packages/nautobot_ssot_aristacv/utils/nautobot.py", line 94, in get_device_version
    version = device.custom_field_data["arista_eos"]
KeyError: 'arista_eos'

After checking further, there is no arista_eos value for the CloudVision device:

>>> for dev in Device.objects.all():
...   print(f"Arista EOS CF for {dev.name} is {dev.custom_field_data['arista_eos'] if dev.custom_field_data.get('arista_eos') else 'MISSING' }")
...
Arista EOS CF for CloudVision is MISSING
Arista EOS CF for nsx01-borderleaf-01 is 4.28.3M
Arista EOS CF for nsx01-leaf-01 is 4.28.5M
Arista EOS CF for nsx01-leaf-02 is 4.28.5M

After manually adding a value to the CloudVision arista_eos field, the problem is solved and subsequent syncs succeeds.

Reference Slack conversation (troubleshoot and debug)

https://networktocode.slack.com/archives/C01NWPK6WHL/p1676042235148109

Environment

• Nautobot version: 1.1.3
• aristacv-sync version: 1.0.2

Proposed Functionality

Is there a possibility to increase amount of roles, as far as I saw in Custom fields, there are supported only “Leafs” or “spines”, we have much more in Arista:
leafs, subleafs, spines, super-spines, borders, route_servers. Is there a possibility to have it also in plugin.

What is more, the role could be associated to Device Role in Nautobot automatically

Use Case