AEtest (Automation Easy Testing) offers a simple and straight-forward way for users to define, execute and debug testcases and testscripts. AEtest is available as a standard component (aetest) in pyATS in an effort to standardize the definition and execution of testcases & testscripts.
AEtest testscript structure is very modular and straightforward. Each testscript is split into three major container sections which are then further broken down into smaller, method sections. These three containers are Common Setup, Testcase(s), and Common Cleanup. Throughout the steps in this section of the lab, you will examine each of these.
In VSCode, using the code keyword in the Terminal window, open a new file for creating a Python pyATS AEtest script:
code -r /home/cisco/Documents/nxapilab/tests/pyats_aetest.py
Like you did in the Python requests section for NX-API, you need to import some modules. In this case we need to import pyATS aetest module and the load module from pyATS Genie.
The logging import is pretty self-explainatory; for logging. And finally, the unicon import is for catching a ConnectionError in subsequent parts of your code.
from pyats import aetest
from genie.testbed import load
import logging
from unicon.core.errors import ConnectionError
logger = logging.getLogger(__name__)
CommonSetup is an optional section within each testscript, defined by inheriting the aetest.CommonSetup class, and declaring one or more Subsections inside. CommonSetup always runs first, and is where all the common configurations, prerequisites and initializations shared between the script's testcases should be performed.
This includes but is not limited to the following:
class CommonSetup(aetest.CommonSetup):
@aetest.subsection
def connect(self, testbed):
"""
establishes connection to all your testbed devices.
"""
# make sure testbed is provided
assert testbed, "Testbed is not provided!"
if len(testbed.devices) == 0:
self.failed('{testbed} is empty'.format(testbed=str(testbed)))
else:
# connect to all testbed devices
# By default ANY error in the CommonSetup will fail the entire test run
# Here we catch common exceptions if a device is unavailable to allow test to continue
try:
for device in testbed:
device.connect(via='rest')
except (TimeoutError, ConnectionError):
logger.error("Unable to connect to all devices")
Testcase is a collection of smaller tests. Testcases are the actual tests to be performed against the device under test. Each testcase may have its own Setup Section and Cleanup Section, and an arbitrary number of smaller Test Sections.
Each Testcase is defined by inheriting aetest.Testcase class, and defining one or more Test Sections inside. Optionally, each Testcase may also have a single Setup Section and a single Cleanup Section. Testcases run in the order as they are defined in the testscript.
class verify_connected(aetest.Testcase):
"""verify_connected
Ensure successful connection to all devices in testbed.
"""
@aetest.test
def test(self, testbed, steps):
# Loop over every device in the testbed
for device_name, device in testbed.devices.items():
with steps.start(
f"Test Connection Status of {device_name}", continue_=True
) as step:
# Test "connected" status
if device.connected:
logger.info(f"{device_name} connected status: {device.connected}")
else:
logger.error(f"{device_name} connected status: {device.connected}")
step.failed()
error_occurred = True
class verify_ospf_process_id(aetest.Testcase):
"""verify_ospf_process_id
"""
@aetest.test
def test(self, testbed, steps):
command = 'show ip ospf neighbors'
# Loop over every device in the testbed
for device_name, device in testbed.devices.items():
with steps.start(
f"Verify OSPF Process ID is UNDERLAY on {device_name}", continue_=True
) as step:
output = device.api.nxapi_method_nxapi_cli('send', command, message_format='json', command_type='cli_show', alias='rest').json()
ospf_process_id = output['ins_api']['outputs']['output']['body']['TABLE_ctx']['ROW_ctx']['ptag']
if ospf_process_id == 'UNDERLAY':
logger.info(f"{device_name} OSPF Process ID: {ospf_process_id}")
else:
logger.error(f"{device_name} OSPF Process ID: {ospf_process_id}")
step.failed()
class verify_ospf_neighbor_count(aetest.Testcase):
"""verify_ospf_neighbor_count
"""
@aetest.test
def test(self, testbed, steps):
command = 'show ip ospf neighbors'
# Loop over every device in the testbed
for device_name, device in testbed.devices.items():
with steps.start(
f"Verify OSPF Neighbors on {device_name}", continue_=True
) as step:
output = device.api.nxapi_method_nxapi_cli('send', command, message_format='json', command_type='cli_show', alias='rest').json()
ospf_nbr_count = output['ins_api']['outputs']['output']['body']['TABLE_ctx']['ROW_ctx']['nbrcount']
if 'spine' in device.name:
expected_ospf_nbr_count = 2
elif 'leaf' in device.name:
expected_ospf_nbr_count = 1
else:
logger.info(f"{device_name} not a spine or leaf!")
step.failed()
if ospf_nbr_count == expected_ospf_nbr_count:
logger.info(f"{device_name} OSPF Neighbor Count: {ospf_nbr_count}")
else:
logger.error(f"{device_name} OSPF Neighbor Count: {ospf_nbr_count}")
step.failed()
class verify_ospf_neighbor_state(aetest.Testcase):
"""verify_ospf_neighbor_state
"""
@aetest.test
def test(self, testbed, steps):
command = 'show ip ospf neighbors'
# Loop over every device in the testbed
for device_name, device in testbed.devices.items():
with steps.start(
f"Verify OSPF Neighbors on {device_name}", continue_=True
) as step:
output = device.api.nxapi_method_nxapi_cli('send', command, message_format='json', command_type='cli_show', alias='rest').json()
ospf_nbrs = output['ins_api']['outputs']['output']['body']['TABLE_ctx']['ROW_ctx']['TABLE_nbr']['ROW_nbr']
if isinstance(ospf_nbrs, list):
for ospf_nbr in ospf_nbrs:
if ospf_nbr['state'] == 'FULL':
logger.info(f"{device_name} OSPF Neighbor State: {ospf_nbr['state']}")
else:
logger.error(f"{device_name} OSPF Neighbor State: {ospf_nbr['state']}")
step.failed()
else:
if ospf_nbr['state'] == 'FULL':
logger.info(f"{device_name} OSPF Neighbor State: {ospf_nbr['state']}")
else:
logger.error(f"{device_name} OSPF Neighbor State: {ospf_nbr['state']}")
step.failed()
CommonCleanup is the last section within each testscript. Any configurations, initializations and environment changes that occured during this script run should be cleaned up or removed here. Eg, the testbed/environment should be returned to the same state as it was before the script run. This includes but is not limited to:
class CommonCleanup(aetest.CommonCleanup):
@aetest.subsection
def disconnect_from_devices(self, testbed):
for device in testbed:
# Only disconnect if we are connected to the device
if device.is_connected() == True:
device.disconnect()
This main function should look similar in comparison to the Python NX-API section, albeit, a bit shorter. The logging level is set first. Next, the AEtest main method is invoked to kickoff the script sections you defined.
if __name__ == '__main__':
# set logger level
logger.setLevel(logging.INFO)
aetest.main()
Executing this script by calling the aetest.main() function is considered a standalone execution. It can be done now, however addtional functionality is gained if ran using the EasyPy Execution method as seen in the next step.
After successfully populating pyats_aetest.py with the above connection information, save your pyats_aetest.py file using Ctrl+s on the Windows keyboard or by clicking File then Save.
Be sure to save your file! Not saving will result in your code not executing.
In VSCode, using the code keyword in the Terminal window, open a new file for creating a Python pyATS EasyPy script:
code -r /home/cisco/Documents/nxapilab/tests/pyats_easypy.py
Add the small snippet of code below into the pyats_easypy.py file that sets the path lookup for the pyats_aetest.py script file:
import os
from pyats.easypy import run
# compute the script path from this location
SCRIPT_PATH = os.path.dirname(__file__)
def main(runtime):
'''job file entrypoint'''
# run script
run(testscript=os.path.join(SCRIPT_PATH, 'pyats_aetest.py'), runtime=runtime)
After successfully populating pyats_easypy.py with the above connection information, save your pyats_easypy.py file using Ctrl+s on the Windows keyboard or by clicking File then Save.
Be sure to save your file! Not saving will result in your code not executing.
Scripts executed with Easypy - Runtime Environment are called EasyPy Execution. In this mode, all environment handling and control is set by the EasyPy launcher. For example, the following features are available:
Execute your AEtest script using EasyPy via the pyATS command line run command, pyats run job to test and verify the expected configuration and state of OSPF.
Notice the --testbed-file option that makes use of the testbed file you defined previously for how to connect to your devices.
pyats run job pyats_easypy.py --testbed-file staging-testbed.yaml --archive-dir=${PWD}/results --no-archive-subdir --no-mail
You will have to scroll up on the Terminal some, but your test script execution should look like the below. Notice there is a summary, Task Result Summary, for each section, i.e. the sections you built out above in your code. Also notice the detailed results, Task Result Details for each testcase run against each device.
2024-02-04T11:39:38: %EASYPY-INFO: --------------------------------------------------------------------------------
2024-02-04T11:39:38: %EASYPY-INFO: Job finished. Wrapping up...
2024-02-04T11:39:39: %EASYPY-INFO: Creating archive file: /home/cisco/Documents/nxapilab/tests/results/pyats_easypy.2024Feb04_11:39:34.118834.zip
2024-02-04T11:39:39: %EASYPY-INFO: +------------------------------------------------------------------------------+
2024-02-04T11:39:39: %EASYPY-INFO: | Easypy Report |
2024-02-04T11:39:39: %EASYPY-INFO: +------------------------------------------------------------------------------+
2024-02-04T11:39:39: %EASYPY-INFO: pyATS Instance : /home/cisco/.pyenv/versions/3.11.6/envs/nxapilab2
2024-02-04T11:39:39: %EASYPY-INFO: Python Version : cpython-3.11.6 (64bit)
2024-02-04T11:39:39: %EASYPY-INFO: CLI Arguments : /home/cisco/.pyenv/versions/nxapilab2/bin/pyats run job pyats_easypy.py --testbed-file staging-testbed.yaml --archive-dir=/home/cisco/Documents/nxapilab/tests/results --no-archive-subdir --no-mail
2024-02-04T11:39:39: %EASYPY-INFO: User : cisco
2024-02-04T11:39:39: %EASYPY-INFO: Host Server : ubuntu-vm-20
2024-02-04T11:39:39: %EASYPY-INFO: Host OS Version : Ubuntu 20.04 focal (x86_64)
2024-02-04T11:39:39: %EASYPY-INFO:
2024-02-04T11:39:39: %EASYPY-INFO: Job Information
2024-02-04T11:39:39: %EASYPY-INFO: Name : pyats_easypy
2024-02-04T11:39:39: %EASYPY-INFO: Start time : 2024-02-04 11:39:37.433250+00:00
2024-02-04T11:39:39: %EASYPY-INFO: Stop time : 2024-02-04 11:39:38.919767+00:00
2024-02-04T11:39:39: %EASYPY-INFO: Elapsed time : 0:00:02
2024-02-04T11:39:39: %EASYPY-INFO: Archive : /home/cisco/Documents/nxapilab/tests/results/pyats_easypy.2024Feb04_11:39:34.118834.zip
2024-02-04T11:39:39: %EASYPY-INFO:
2024-02-04T11:39:39: %EASYPY-INFO: Total Tasks : 1
2024-02-04T11:39:39: %EASYPY-INFO:
2024-02-04T11:39:39: %EASYPY-INFO: Overall Stats
2024-02-04T11:39:39: %EASYPY-INFO: Passed : 6
2024-02-04T11:39:39: %EASYPY-INFO: Passx : 0
2024-02-04T11:39:39: %EASYPY-INFO: Failed : 0
2024-02-04T11:39:39: %EASYPY-INFO: Aborted : 0
2024-02-04T11:39:39: %EASYPY-INFO: Blocked : 0
2024-02-04T11:39:39: %EASYPY-INFO: Skipped : 0
2024-02-04T11:39:39: %EASYPY-INFO: Errored : 0
2024-02-04T11:39:39: %EASYPY-INFO:
2024-02-04T11:39:39: %EASYPY-INFO: TOTAL : 6
2024-02-04T11:39:39: %EASYPY-INFO:
2024-02-04T11:39:39: %EASYPY-INFO: Success Rate : 100.00 %
2024-02-04T11:39:39: %EASYPY-INFO:
2024-02-04T11:39:39: %EASYPY-INFO: Section Stats
2024-02-04T11:39:39: %EASYPY-INFO: Passed : 6
2024-02-04T11:39:39: %EASYPY-INFO: Passx : 0
2024-02-04T11:39:39: %EASYPY-INFO: Failed : 0
2024-02-04T11:39:39: %EASYPY-INFO: Aborted : 0
2024-02-04T11:39:39: %EASYPY-INFO: Blocked : 0
2024-02-04T11:39:39: %EASYPY-INFO: Skipped : 0
2024-02-04T11:39:39: %EASYPY-INFO: Errored : 0
2024-02-04T11:39:39: %EASYPY-INFO:
2024-02-04T11:39:39: %EASYPY-INFO: TOTAL : 6
2024-02-04T11:39:39: %EASYPY-INFO:
2024-02-04T11:39:39: %EASYPY-INFO: Section Success Rate : 100.00 %
2024-02-04T11:39:39: %EASYPY-INFO:
2024-02-04T11:39:39: %EASYPY-INFO: +------------------------------------------------------------------------------+
2024-02-04T11:39:39: %EASYPY-INFO: | Task Result Summary |
2024-02-04T11:39:39: %EASYPY-INFO: +------------------------------------------------------------------------------+
2024-02-04T11:39:39: %EASYPY-INFO: Task-1: pyats_aetest.common_setup PASSED
2024-02-04T11:39:39: %EASYPY-INFO: Task-1: pyats_aetest.verify_connected PASSED
2024-02-04T11:39:39: %EASYPY-INFO: Task-1: pyats_aetest.verify_ospf_process_id PASSED
2024-02-04T11:39:39: %EASYPY-INFO: Task-1: pyats_aetest.verify_ospf_neighbor_count PASSED
2024-02-04T11:39:39: %EASYPY-INFO: Task-1: pyats_aetest.verify_ospf_neighbor_state PASSED
2024-02-04T11:39:39: %EASYPY-INFO: Task-1: pyats_aetest.common_cleanup PASSED
2024-02-04T11:39:39: %EASYPY-INFO:
2024-02-04T11:39:39: %EASYPY-INFO: +------------------------------------------------------------------------------+
2024-02-04T11:39:39: %EASYPY-INFO: | Task Result Details |
2024-02-04T11:39:39: %EASYPY-INFO: +------------------------------------------------------------------------------+
2024-02-04T11:39:39: %EASYPY-INFO: Task-1: pyats_aetest
2024-02-04T11:39:39: %EASYPY-INFO: |-- common_setup PASSED
2024-02-04T11:39:39: %EASYPY-INFO: | `-- connect PASSED
2024-02-04T11:39:39: %EASYPY-INFO: |-- verify_connected PASSED
2024-02-04T11:39:39: %EASYPY-INFO: | `-- test PASSED
2024-02-04T11:39:39: %EASYPY-INFO: | |-- STEP 1: Test Connection Status of staging-spine1 PASSED
2024-02-04T11:39:39: %EASYPY-INFO: | |-- STEP 2: Test Connection Status of staging-leaf1 PASSED
2024-02-04T11:39:39: %EASYPY-INFO: | `-- STEP 3: Test Connection Status of staging-leaf2 PASSED
2024-02-04T11:39:39: %EASYPY-INFO: |-- verify_ospf_process_id PASSED
2024-02-04T11:39:39: %EASYPY-INFO: | `-- test PASSED
2024-02-04T11:39:39: %EASYPY-INFO: | |-- STEP 1: Verify OSPF Process ID is UNDERLAY on staging-s... PASSED
2024-02-04T11:39:39: %EASYPY-INFO: | |-- STEP 2: Verify OSPF Process ID is UNDERLAY on staging-l... PASSED
2024-02-04T11:39:39: %EASYPY-INFO: | `-- STEP 3: Verify OSPF Process ID is UNDERLAY on staging-l... PASSED
2024-02-04T11:39:39: %EASYPY-INFO: |-- verify_ospf_neighbor_count PASSED
2024-02-04T11:39:39: %EASYPY-INFO: | `-- test PASSED
2024-02-04T11:39:39: %EASYPY-INFO: | |-- STEP 1: Verify OSPF Neighbors on staging-spine1 PASSED
2024-02-04T11:39:39: %EASYPY-INFO: | |-- STEP 2: Verify OSPF Neighbors on staging-leaf1 PASSED
2024-02-04T11:39:39: %EASYPY-INFO: | `-- STEP 3: Verify OSPF Neighbors on staging-leaf2 PASSED
2024-02-04T11:39:39: %EASYPY-INFO: |-- verify_ospf_neighbor_state PASSED
2024-02-04T11:39:39: %EASYPY-INFO: | `-- test PASSED
2024-02-04T11:39:39: %EASYPY-INFO: | |-- STEP 1: Verify OSPF Neighbors on staging-spine1 PASSED
2024-02-04T11:39:39: %EASYPY-INFO: | |-- STEP 2: Verify OSPF Neighbors on staging-leaf1 PASSED
2024-02-04T11:39:39: %EASYPY-INFO: | `-- STEP 3: Verify OSPF Neighbors on staging-leaf2 PASSED
2024-02-04T11:39:39: %EASYPY-INFO: `-- common_cleanup PASSED
2024-02-04T11:39:39: %EASYPY-INFO: `-- disconnect_from_devices PASSED
2024-02-04T11:39:40: %EASYPY-INFO: Done!
Once executed, close pyats_aetest.py and pyats_easypy.py by clicking the small x (close button) to the right of the file tab:
Once completed, continue to the next section to examine pyATS testing using the RobotFramework.