Implement "AntaTest" and "AntaAsyncTest" as protocols about anta HOT 3 OPEN

Thanks for raising this :)
Implementing tests as coroutines is definitely needed for this framework to scale.
Do you have any example of a test that would require to be run sequentially ?

from anta.

Thanks for raising this :) Implementing tests as coroutines is definitely needed for this framework to scale. Do you have any example of a test that would require to be run sequentially ?

I think it could be useful for large deployments, or where there are a vast number of tests with high correlation. For example, some BGP overlay session tests wouldn't make sense if the loopback is not defined, therefore you could cluster tests together, running the tests inside the cluster as co-routines and separate cluster tests in sequence. For instance:

• Test group 1:
  • test_function_1
  • test_function_2
  • test_function_3
• Test group 2:
  • test_function_4
  • test_function_5
  • test_function_6

Where usually the functions in test_group1 need to pass in order to make sense out of the tests in group 2, etc.

async def run_sequence(*functions: Awaitable[Any]) -> None:
  for function in functions:
    await function

async def run_parallel(*functions: Awaitable[Any]) -> None:
  await asyncio.gather(*functions)

await run_sequence(
  run_parallel(
    test_function_1(),
    test_function_2(),
    test_function_3()
  ),
  run_parallel(
    test_function_4(),
    test_function_5(),
    test_function_6()
  )
)

Other use cases could be managing how tests run across the inventory of devices (e.g., first spines, then the leafs, etc). This kind of flexibility may allow to easily optimize performance based on the inventory and test catalog...

from anta.

PR #122 implements asyncio in ANTA. However, we still cannot define a test hierarchy/dependency if some of them needs to be run sequentially: everything runs in parallel today.
However, we have written the anta_test decorator to factorise code that is identical across all tests.

from anta.