arista-netdevops-community / anta Goto Github PK
View Code? Open in Web Editor NEWWhat do you call an ant with frogs legs?
Home Page: https://www.anta.ninja/
License: Apache License 2.0
What do you call an ant with frogs legs?
Home Page: https://www.anta.ninja/
License: Apache License 2.0
and then remove unused-argument from pylintrc bypass configuration
create a test_catalog variable somewhere to map tests name and tests id
the tests id can have a format like xx.xx where the first 2 digits will be the category (BGP, BFD, ...) and the last 2 digits will be to identify the test itself.
Errors:
/home/cbaillar/git_projects/anta/anta/decorators.py:78: RuntimeWarning: coroutine 'AntaTest.collect' was never awaited anta_test.collect() RuntimeWarning: Enable tracemalloc to get the object allocation traceback [13:12:56] ERROR Error when running tests: TypeError: argument of type 'NoneType' is not iterable
BGP check_bgp_family_enable
decorator returns None and anta_test.collect() is not awaited.
Add support for more than one template per test.
Example:
template = AntaTestTemplate(template="test aaa group tacacs+ {user} {pass}", ofmt="text")
When running the following command, CLI crashes and return the following stack-trace
Command
anta exec snapshot -c .personal/snapshot.yml
Snapshot file
json_format:
- show version
text_format:
- show lldp neighbors
Error
ERROR Could not collect commands on device spine01
DEBUG Exception raised for device spine01 - KeyError: 'data'
DEBUG Traceback (most recent call last):
File "/Users/tgrimonet/Projects/arista-community/\
anta/anta/cli/exec/utils.py", line 69, in collect
result = await dev.session.cli(
File "/Users/tgrimonet/.pyenv/versions/3.9.9/envs/\
anta/lib/python3.9/site-packages/aioeapi/device.py",\
line 183, in cli
res = await self.jsonrpc_exec(jsonrpc)
File "/Users/tgrimonet/.pyenv/versions/3.9.9/envs/\
anta/lib/python3.9/site-packages/aioeapi/device.py", \
line 250, in jsonrpc_exec
cmd_data = err_data["data"]
KeyError: 'data'
Information
3.9.9
0.5.0
and devel
Today we are using decorators on tests that should be skipped. We should leverage AntaTestFilter class instead for more flexibility.
Ex:
class VerifyBGPIPv4UnicastCount(AntaTest):
test_filters = [SkipBGPFilter(family="ipv4")]
class SkipBGPFilter(AntaTestFilter):
def __init__(self, family: str) -> None:
"""
Class initialization with the BGP address family to verify.
"""
self.family = family
There is already a verify_running_config_diffs. It would be nice to introduce a verify_config_compliance to check the running or startup config matches an expected or golden config.
Help message
nta exec
######## #
################# #
###################### #
######################### #
############################
##############################
###############################
###############################
##############################
# ######## #
## ### #### ##
### ###
#### ###
#### ########## ####
####################### ####
#################### ####
################## ####
############ ##
######## ###
######### #####
############ ######
######## #########
##### ########
### #########
###### ############
#######################
# # ### # # ##
########################
## ## ## ##
https://www.anta.ninja
Usage: anta exec [OPTIONS] COMMAND [ARGS]...
Execute commands to inventory devices
Options:
--help Show this message and exit.
Commands:
clear-counters Clear counter statistics on EOS devices
collect-tech-support Collect scheduled tech-support from eos devices.
snapshot Collect commands output from devices in inventory
Autocompletion
eval "$(_ANTA_COMPLETE=zsh_source anta)" > /dev/null
(eval):31: no such file or directory: https://www.anta.ninja
sh port-channel load-balance
In order to allow users to build their own test collection, it would be nice to add option to support catalog pointing to custom libraries.
Today, we only look at anta.tests
package but we could extend YAML format to support both anta.tests
and a custom package with the following format:
<package.name>:
- <test function>:
<function options>
An example would be:
anta.tests.software:
- verify_eos_version: # Verifies the device is running one of the allowed EOS version.
versions: # List of allowed EOS versions.
- 4.25.4M
- 4.26.1F
inetsix.tests.mydesign:
- check_topology:
anta.loader.parse_catalog
Hi,
Wondering if we can have network ranges specified on the devices.txt rather than using unique ip?
Thanks
Hari
This will require updates in the md files
remove enable password from code to test devices (check-devices.py and tests_eos/functions.py)?
Intermittent errors when running some of the logging test cases that have complex commands.
ERROR Command failed on NW-CORE: CLI command 2 of 2 'show logging | awk '/Trap logging/,/Sequence models.py:239
numbers/{if(/Sequence numbers/) exit; print}'' failed: internal error
I am having an issue running the scripts, as jsonrpclib doesn't work with Python3:
testvm:~/network_tests_automation# ./check-devices-reachability.py --help
Traceback (most recent call last):
File "./check-devices-reachability.py", line 5, in <module>
from jsonrpclib import Server
ModuleNotFoundError: No module named 'jsonrpclib'
I've installed PIP dependencies as instructed, and even added 'pip install jsonrpclib' separately:
test-vm:~/network_tests_automation# /usr/bin/env python3 -V
Python 3.6.9
testvm:~# pip list | grep jsonrpc
jsonrpclib (0.1.7)
jsonrpclib-pelix (0.4.3.2)
There seems to be a known issue with this library and Python3: https://stackoverflow.com/questions/38664102/python-jsonrpclib-not-working-after-upgrade-to-python-3-5-2
Any ideas how to get around this?
Sharing some ideas that could improve the framework... Need a way to enforce the format of the test functions to ensure all functions are written correctly (return boolean, etc) and the user can introduce more functions on runtime. Both normal/sequential and async modes could be supported, running the async tests in parallel on a "AntaAsyncTestRunner" and the other functions in a "AntaTestRunner". A test manager could handle the runners and dispatch the tests to each of them delegating the execution. So having the tests modelled in a flexible way could be the first step towards a more generic execution framework that can be customized/extended by the user on runtime.
Currently, enable password is passed as argument when starting ANTA, stored as an attribute in the InventoryDevice model, and used in the test code to enter in privileged mode when needed.
This issue is to explore if we could move this logic in the AntaInventory class, add an attribute (e.g. device.privileged
) in the InventoryDevice model to define if the device session has been established with privileged mode (similar to established
and is_online
).
This way, if a test needs privileged mode, we could have a decorator to check the device.privileged
attribute and remove all the logic in coroutines that uses the enable password.
The module is too large (>1600 lines).
we can split it into diff modules:
#!/usr/bin/env python3
## This script collects all the tech-support files stored on Arista switches flash and copies them locally
import ssl
import paramiko
from jsonrpclib import Server
from scp import SCPClient
from time import strftime, gmtime
from argparse import ArgumentParser
from getpass import getpass
import os
from sys import exit
from socket import setdefaulttimeout
port = 22
ssl._create_default_https_context = ssl._create_unverified_context
date = strftime("%d %b %Y %H:%M:%S", gmtime())
def device_directories (device, root_dir):
cwd = os.getcwd()
show_tech_directory = cwd + '/' + root_dir
device_directory = show_tech_directory + '/' + device
for directory in [show_tech_directory, device_directory]:
if not os.path.exists(directory):
os.makedirs(directory)
result = show_tech_directory, device_directory
return result
def createSSHClient (device, port, username, password):
client = paramiko.SSHClient()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect(device, port, username, password)
return client
def main():
parser = ArgumentParser(
description='Collect all the tech-support files'
)
parser.add_argument(
'-i',
help='Text file containing a list of switches',
dest='file',
required=True
)
parser.add_argument(
'-u',
help='Devices username',
dest='username',
required=True
)
parser.add_argument(
'-o',
help='Output directory',
dest='output_directory',
required=True
)
args = parser.parse_args()
args.password = getpass(prompt='Device password: ')
try:
with open(args.file, 'r') as file:
devices = file.readlines()
except:
print('Error opening ' + args.file)
exit(1)
for i,device in enumerate(devices):
devices[i] = device.strip()
# Deletes unreachable devices from devices list
unreachable = []
print('Connecting to devices .... please be patient ... ')
for device in devices:
try:
setdefaulttimeout(5)
url = 'https://%s:%s@%s/command-api' %(args.username, args.password, device)
switch = Server(url)
response = switch.runCmds(1, ['enable'])
except:
unreachable.append(device)
for item in unreachable:
devices.remove(item)
print("Can not connect to device " + item)
# Collects all the tech-support files stored on Arista switches flash and copies them locally
for device in devices:
url = "https://" + args.username + ":" + args.password + "@" + device + "/command-api"
try:
## Create one zip file named all_files.zip on the device with the all the show tech-support files in it
cmds=['enable','bash timeout 30 zip /mnt/flash/schedule/all_files.zip /mnt/flash/schedule/tech-support/*']
switch = Server(url)
switch.runCmds(1,cmds, 'text')
## Connect to the device using SSH
ssh = createSSHClient(device, port, args.username, args.password)
## Create directories
output_dir = device_directories (device, args.output_directory)
## Get the zipped file all_files.zip using SCP and save it locally
my_path = output_dir[1] + '/' + date + '.zip'
ssh = createSSHClient(device, port, args.username, args.password)
scp = SCPClient(ssh.get_transport())
scp.get("/mnt/flash/schedule/all_files.zip",local_path = my_path)
scp.close()
## Delete the created zip file on the device
cmds=['enable','bash timeout 30 rm /mnt/flash/schedule/all_files.zip']
switch.runCmds(1,cmds, 'text')
except:
print('You are unlucky today! ' + device + ' does not like this script')
if __name__ == '__main__':
main()
```
The --by-host
knob is broken and returns a trace:
$ check-devices.py -i anta-inventory.yml -c tests-bases.yml -u xxx -p xxx --by-host --table
Traceback (most recent call last):
File "/home/tom/.pyenv/versions/arista-avd-3.9/bin/check-devices.py", line 7, in <module>
exec(compile(f.read(), __file__, 'exec'))
File "/home/tom/Projects/arista/network-test-automation/scripts/check-devices.py", line 312, in <module>
reporter.report_summary_hosts(
File "/home/tom/Projects/arista/network-test-automation/anta/reporter/__init__.py", line 251, in report_summary_hosts
for host_read in result_manager.get_hosts():
File "/home/tom/Projects/arista/network-test-automation/anta/result_manager/__init__.py", line 197, in get_hosts
if str(testcase.host) not in result_list:
AttributeError: 'TestResult' object has no attribute 'host'
This option was working in v0.3.0 and is a regression part of one of these PRs:
Then remove duplicate-code from pylintrc bypass configuration
Hello,
Currently the only precision about the tests is the name of the function itself. It is therefore necessary to read the code of the function to understand how it works and to have more details.
An idea would be to add to each test function a python decorator with two attributes: label and description.
Example with this sample piece of code:
def decorator(**kwargs):
def wrapper(function):
for k, v in kwargs.items():
setattr(function, k, v)
return function
return wrapper
@decorator(label='EOS version', description='Verifies the device is running one of the allowed EOS version')
def verify_eos_version():
pass
print(f'label: {verify_eos_version.label}\ndescription: {verify_eos_version.description}')
>>> label: EOS version
>>> description: Verifies the device is running one of the allowed EOS version
Now, we can access easily this information, without having to add two fields in a .yaml
file, for example.
Thank you very much.
req an update of setup file
The minimal Python version is now 3.10 with aio-eapi v0.6.0
use the command show logging informational last 7 days
in text
format
and search for keywords (variables used in function arguments) in the output like as example TRANSCEIVER_REMOVED
or EVPN-3-BLACKLISTED_DUPLICATE_MAC
or UPDOWN
and return True if a keyword is NOT in the output
keywords = ["EVPN-3-BLACKLISTED_DUPLICATE_MAC", "TRANSCEIVER_REMOVED", "UPDOWN"]
def verify_keyword_not_in_logs(device, keywords):
response = device.runCmds(1, ["show logging informational last 7 days"], 'text')
if response[0]['output'] != "":
data = response[0]['output'].split('\n')
for keyword in keywords:
for line in data:
if keyword in line:
return False
return True
making the count (number arg) optionnal
Increase flexibility by using tags on inventory devices and the test catalog, so that catalog tests can run on relevant devices only. For instance, in s typical EVPN scenario there.might be some tests that should run only in spines and some other tests only on leaf switches, having also some tests that run on all devices. Using tags or filters on tests and inventory devices could achieve this easily and add flexibility to the way tests execute. For inventory networks/ranges, tags would apply to the whole range/network...
Sharing some ideas that could improve the framework... Need a way to enforce the format of the test functions to ensure all functions are written correctly (return boolean, etc) and the user can introduce more functions on runtime. Both normal/sequential and async modes could be supported, running the async tests in parallel on a "AntaAsyncTestRunner" and the other functions in a "AntaTestRunner". A test manager could handle the runners and dispatch the tests to each of them delegating the execution. So having the tests modelled in a flexible way could be the first step towards a more generic execution framework that can be customized/extended by the user on runtime.
getting CVP inventory can be done using REST API or Resource API or cvprac.
to be able to give other Device Classes
requires an interface for what we expect from a device
e.g:
13.01. verify_spanning_tree_blocked_ports
leaf-01: Fail
The following ports are blocked by STP:
Ethernet3
Ethernet4
Ethernet5
sh ip ospf neighbor | exclude FULL
count established neighbors using the arg VRF (all, default, vrf1, ...)
and remove from this repo the scripts not related to the tests library
Wrong tests showed (Security) in the Logging tests section.
https://www.anta.ninja/api/tests.logging/
Fixed in #205
Add category and description to TestResult
workflow to:
One of the main concerns for usability of the framework is performance and the fact that tests are run in sequence for each inventory device. Most of the tests are RPC calls over the network and creating connections that cause delays for am inventory of hundreds of devices. Would be nice to have an async method where all the tests (or just a subset) can run in parallel and waiting time could be used efficiently.
Also async inventory could initiate connection to all inventory hosts in parallel.
Add a CLI entrypoint to convert a YAML ansible inventory from AVD to ANTA
anta get from-avd <path to yaml file> --container-name fabric
Is it possible when using ANTA to collect the tech-support filles from a switch in the /mnt/flash/schedule/tech-support folder to only collect the latest file and not all the files?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.