RobotFramework is an open source, Python-based test automation framework that is commonly used for acceptance test automation using English-like keyword-driven tests. pyATS and RobotFramework can be easily integrated to gain the following features:
pyats[full] package install.
Verify the RobotFramework package and associated pyATS packages are installed in your virtualenv.
pip freeze | grep robot
genie.libs.robot==24.1
pyats.robot==24.1
robotframework==7.0
You can verify the specific RobotFramework version also by using the command line for Robot, robot:
robot --version
Robot Framework 7.0 (Python 3.11.6 on linux)
In VSCode, using the code keyword in the Terminal window, open a new file to create a Robot file:
code -r /home/cisco/Documents/nxapilab/tests/pyats.robot
There is a huge community of contributors around the tool which users can leverage to extend it's use cases for various needs. External libraries can be installed based on the user's needs or new custom libraries can be created with ease.
Test libraries are normally imported using the Library keyword in the Setting section and having the library name in the
subsequent column. To use pyATS and pyATS Genie keywords in your Robot testcase script, you import ats.robot.pyATSRobot,
genie.libs.robot.GenieRobot, and genie.libs.robot.GenieRobotApis.
Respective keywords that are available from each pyATS library can be located here:
A Suite Setup is executed before running any of the suite's test cases or child test suites, and a Suite Teardown is executed
after them. The Suite Setup you will implement uses the testbed yaml file you created and used in your Python-based pyATS script by using the use testbed pyATS keyword.
It connects to all devices using the pyATS connect to all devices keyword, and then finally calls a user custom keyword, Set Fabric Env that you will add to the Robot file in the next step.
*** Settings ***
Library ats.robot.pyATSRobot
Library genie.libs.robot.GenieRobot
Library genie.libs.robot.GenieRobotApis
# Load topology/testbed yaml file
Suite Setup Run Keywords
... use testbed "%{TESTBED}" AND
... connect to all devices AND
... Set Fabric Env
# Disconnect sessions from all devices
Suite Teardown Run Keywords
... disconnect from all devices
The Keywords section is used to create new keywords by combining existing keywords together.
These keywords are called user keywords to differentiate them from existing library keywords that are
implemented in test libraries, again, such as the pyATS library keywords you imported in the previous step.
In the Keywords section, you will add the following keywords for use in Suite Setup and the Testcases section covered in the next step:
Set Fabric Env: This keyword is leveraged in the Suite Setup to select the correct testbed yaml file. To this point in the lab you have created the testbed yaml for your staging (development) fabric, but later in the lab, you will create the testbed yaml for your prod fabric.Get BGP L2VPN Summary From Device: This keyword simply is a "getter" keyword that gets the data from the device and is to be used in other keywords, such as the subsequent verify keywords.Verify BGP ASN For All Devices: This keyword calls the Get BGP L2VPN Summary From Device user keyword and verifies the expected BGP ASN based on user input.Verify BGP Neighbor Count: This keyword calls the Get BGP L2VPN Summary From Device user keyword and verifies the expected number of BGP neighbors based on user input.
Within each of these user keywords, keywords from Robot's core library are used to perform the validation checks. These include setting variables with Set Variable,
FOR loops, Run Keyword If...Else, and comparision checks with keywords such as Should Be Equal As Numbers.
*** Keywords ***
Set Fabric Env
${is_prod}= Evaluate "prod" in """%{TESTBED}"""
${is_staging}= Evaluate "staging" in """%{TESTBED}"""
@{DEVICE_LIST}= Run Keyword If ${is_prod} Create list prod-spine1 prod-leaf1 prod-leaf2
... ELSE Create list staging-spine1 staging-leaf1 staging-leaf2
Set Suite Variable ${DEVICE_LIST}
Get BGP L2VPN Summary From Device
[Arguments] ${device}
${output}= nxapi method nxapi cli device=${device} action=send commands=show bgp l2vpn evpn summary message_format=json_rpc command_type=cli alias=rest
Log ${output.json()}
RETURN ${output.json()}
Verify BGP ASN For All Devices
[Arguments] ${device_list} ${expected_bgp_asn}
FOR ${device} IN @{device_list}
${output}= Get BGP L2VPN Summary From Device device=${device}
${bgp_asn}= Set Variable ${output}[result][body][TABLE_vrf][ROW_vrf][vrf-local-as]
Should Be Equal As Numbers ${bgp_asn} ${expected_bgp_asn}
END
Verify BGP Neighbor Count
[Arguments] ${device_list} ${expected_bgp_nbr_count}
FOR ${device} IN @{device_list}
${output}= Get BGP L2VPN Summary From Device device=${device}
${bgp_nbrs}= Set Variable ${output}[result][body][TABLE_vrf][ROW_vrf][TABLE_af][ROW_af][TABLE_saf][ROW_saf][TABLE_neighbor][ROW_neighbor]
${type string}= Evaluate type($bgp_nbrs).__name__
${bgp_nbr_count}= Run Keyword If ${type string} == list Get length ${bgp_nbrs}
... ELSE Set Variable 1
Should Be Equal As Numbers ${bgp_nbr_count} ${expected_bgp_nbr_count}
END
Test cases are constructed in the Test Cases sections. Keywords can be imported from test
libraries, resource files, or created in the keyword section of the test case file itself as you have done above for simplicity.
As you grow your own keyword repositories, then it would be best practice to explore creating your own libraries or resource files to import.
*** Test Cases ***
VERIFY ALL DEVICE'S EXPECTED LOCAL BGP ASN via NX-API CLI JSON-RPC
Verify BGP ASN For All Devices device_list=${DEVICE_LIST} expected_bgp_asn=65001
VERIFY SPINE DEVICE(S) EXPECTED NEIGHBOR COUNT via NX-API CLI JSON-RPC
Verify BGP Neighbor Count device_list=${DEVICE_LIST[:1]} expected_bgp_nbr_count=2
VERIFY LEAF DEVICE(S) EXPECTED NEIGHBOR COUNT via NX-API CLI JSON-RPC
Verify BGP Neighbor Count device_list=${DEVICE_LIST[1:2]} expected_bgp_nbr_count=1
After successfully populating pyats.robot with the above sections, save your pyats.robot file using Ctrl+s on the Windows keyboard or by clicking File then Save.
Be sure to save your file! Not saving will result in your code not executing.
The Robot script will be executed using the pyATS EasyPy execution method for additional functionality as noted in the previous section.
pyats run robot pyats.robot --testbed-file staging-testbed.yaml --archive-dir=${PWD}/results --no-archive-subdir --no-mai
You will have to scroll up on the Terminal some, but your Robot script execution should look like the below. Notice the test cases are toward the top and they should show as PASS. Additionally, there is the same EasyPy reporting to show a summary and detail view of the test execution steps.
2024-02-04T11:53:34: %EASYPY-INFO: Starting task execution: Task-1
2024-02-04T11:53:34: %EASYPY-INFO: test harness = pyats.robot.runner
2024-02-04T11:53:34: %EASYPY-INFO: testscript = /home/cisco/Documents/nxapilab/tests/pyats.robot
==============================================================================
Pyats
==============================================================================
VERIFY ALL DEVICE'S EXPECTED LOCAL BGP ASN via NX-API CLI JSON-RPC | PASS |
------------------------------------------------------------------------------
VERIFY SPINE DEVICE(S) EXPECTED NEIGHBOR COUNT via NX-API CLI JSON... | PASS |
------------------------------------------------------------------------------
VERIFY LEAF DEVICE(S) EXPECTED NEIGHBOR COUNT via NX-API CLI JSON-RPC | PASS |
------------------------------------------------------------------------------
Pyats | PASS |
3 tests, 3 passed, 0 failed
==============================================================================
Output: /home/cisco/.pyats/runinfo/pyats.2024Feb04_11:53:30.655199/Task-1.robot/output.xml
Log: /home/cisco/.pyats/runinfo/pyats.2024Feb04_11:53:30.655199/Task-1.robot/log.html
Report: /home/cisco/.pyats/runinfo/pyats.2024Feb04_11:53:30.655199/Task-1.robot/report.html
2024-02-04T11:54:05: %CONTRIB-INFO: WebEx Token not given as argument or in config. No WebEx notification will be sent
2024-02-04T11:54:05: %EASYPY-INFO: --------------------------------------------------------------------------------
2024-02-04T11:54:05: %EASYPY-INFO: Job finished. Wrapping up...
2024-02-04T11:54:06: %EASYPY-INFO: Creating archive file: /home/cisco/Documents/nxapilab/tests/results/pyats.2024Feb04_11:53:30.655199.zip
2024-02-04T11:54:06: %EASYPY-INFO: +------------------------------------------------------------------------------+
2024-02-04T11:54:06: %EASYPY-INFO: | Easypy Report |
2024-02-04T11:54:06: %EASYPY-INFO: +------------------------------------------------------------------------------+
2024-02-04T11:54:06: %EASYPY-INFO: pyATS Instance : /home/cisco/.pyenv/versions/3.11.6/envs/nxapilab2
2024-02-04T11:54:06: %EASYPY-INFO: Python Version : cpython-3.11.6 (64bit)
2024-02-04T11:54:06: %EASYPY-INFO: CLI Arguments : /home/cisco/.pyenv/versions/nxapilab2/bin/pyats run robot pyats.robot --testbed-file staging-testbed.yaml --archive-dir=/home/cisco/Documents/nxapilab/tests/results --no-archive-subdir --no-mai
2024-02-04T11:54:06: %EASYPY-INFO: User : cisco
2024-02-04T11:54:06: %EASYPY-INFO: Host Server : ubuntu-vm-20
2024-02-04T11:54:06: %EASYPY-INFO: Host OS Version : Ubuntu 20.04 focal (x86_64)
2024-02-04T11:54:06: %EASYPY-INFO:
2024-02-04T11:54:06: %EASYPY-INFO: Job Information
2024-02-04T11:54:06: %EASYPY-INFO: Name : pyats
2024-02-04T11:54:06: %EASYPY-INFO: Start time : 2024-02-04 11:53:33.992889+00:00
2024-02-04T11:54:06: %EASYPY-INFO: Stop time : 2024-02-04 11:54:05.506023+00:00
2024-02-04T11:54:06: %EASYPY-INFO: Elapsed time : 0:00:32
2024-02-04T11:54:06: %EASYPY-INFO: Archive : /home/cisco/Documents/nxapilab/tests/results/pyats.2024Feb04_11:53:30.655199.zip
2024-02-04T11:54:06: %EASYPY-INFO:
2024-02-04T11:54:06: %EASYPY-INFO: Total Tasks : 1
2024-02-04T11:54:06: %EASYPY-INFO:
2024-02-04T11:54:06: %EASYPY-INFO: Overall Stats
2024-02-04T11:54:06: %EASYPY-INFO: Passed : 3
2024-02-04T11:54:06: %EASYPY-INFO: Passx : 0
2024-02-04T11:54:06: %EASYPY-INFO: Failed : 0
2024-02-04T11:54:06: %EASYPY-INFO: Aborted : 0
2024-02-04T11:54:06: %EASYPY-INFO: Blocked : 0
2024-02-04T11:54:06: %EASYPY-INFO: Skipped : 0
2024-02-04T11:54:06: %EASYPY-INFO: Errored : 0
2024-02-04T11:54:06: %EASYPY-INFO:
2024-02-04T11:54:06: %EASYPY-INFO: TOTAL : 3
2024-02-04T11:54:06: %EASYPY-INFO:
2024-02-04T11:54:06: %EASYPY-INFO: Success Rate : 100.00 %
2024-02-04T11:54:06: %EASYPY-INFO:
2024-02-04T11:54:06: %EASYPY-INFO: Section Stats
2024-02-04T11:54:06: %EASYPY-INFO: Passed : 3
2024-02-04T11:54:06: %EASYPY-INFO: Passx : 0
2024-02-04T11:54:06: %EASYPY-INFO: Failed : 0
2024-02-04T11:54:06: %EASYPY-INFO: Aborted : 0
2024-02-04T11:54:06: %EASYPY-INFO: Blocked : 0
2024-02-04T11:54:06: %EASYPY-INFO: Skipped : 0
2024-02-04T11:54:06: %EASYPY-INFO: Errored : 0
2024-02-04T11:54:06: %EASYPY-INFO:
2024-02-04T11:54:06: %EASYPY-INFO: TOTAL : 3
2024-02-04T11:54:06: %EASYPY-INFO:
2024-02-04T11:54:06: %EASYPY-INFO: Section Success Rate : 100.00 %
2024-02-04T11:54:06: %EASYPY-INFO:
2024-02-04T11:54:06: %EASYPY-INFO: +------------------------------------------------------------------------------+
2024-02-04T11:54:06: %EASYPY-INFO: | Task Result Summary |
2024-02-04T11:54:06: %EASYPY-INFO: +------------------------------------------------------------------------------+
2024-02-04T11:54:06: %EASYPY-INFO: Task-1: pyats.VERIFY ALL DEVICE'S EXPECTED LOCAL BGP ASN via NX-API... PASSED
2024-02-04T11:54:06: %EASYPY-INFO: Task-1: pyats.VERIFY SPINE DEVICE(S) EXPECTED NEIGHBOR COUNT via NX... PASSED
2024-02-04T11:54:06: %EASYPY-INFO: Task-1: pyats.VERIFY LEAF DEVICE(S) EXPECTED NEIGHBOR COUNT via NX-... PASSED
2024-02-04T11:54:06: %EASYPY-INFO:
2024-02-04T11:54:06: %EASYPY-INFO: +------------------------------------------------------------------------------+
2024-02-04T11:54:06: %EASYPY-INFO: | Task Result Details |
2024-02-04T11:54:06: %EASYPY-INFO: +------------------------------------------------------------------------------+
2024-02-04T11:54:06: %EASYPY-INFO: Task-1: pyats
2024-02-04T11:54:06: %EASYPY-INFO: |-- VERIFY ALL DEVICE'S EXPECTED LOCAL BGP ASN via NX-API CLI JSON-RPC PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | `-- 1_Verify BGP ASN For All Devices PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 2_${device} IN @{device_list} PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 3_${device} = staging-spine1 PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 4_Get BGP L2VPN Summary From Device PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 5_Nxapi Method Nxapi Cli PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 6_Log PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 7_ PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 8_Set Variable PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 9_Should Be Equal As Numbers PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 10_${device} = staging-leaf1 PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 11_Get BGP L2VPN Summary From Device PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 12_Nxapi Method Nxapi Cli PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 13_Log PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 14_ PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 15_Set Variable PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 16_Should Be Equal As Numbers PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 17_${device} = staging-leaf2 PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 18_Get BGP L2VPN Summary From Device PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 19_Nxapi Method Nxapi Cli PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 20_Log PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 21_ PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 22_Set Variable PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | `-- 23_Should Be Equal As Numbers PASSED
2024-02-04T11:54:06: %EASYPY-INFO: |-- VERIFY SPINE DEVICE(S) EXPECTED NEIGHBOR COUNT via NX-API CLI J... PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | `-- 1_Verify BGP Neighbor Count PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 2_${device} IN @{device_list} PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 3_${device} = staging-spine1 PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 4_Get BGP L2VPN Summary From Device PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 5_Nxapi Method Nxapi Cli PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 6_Log PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 7_ PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 8_Set Variable PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 9_Evaluate PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 10_Run Keyword If PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | |-- 11_Get Length PASSED
2024-02-04T11:54:06: %EASYPY-INFO: | `-- 12_Should Be Equal As Numbers PASSED
2024-02-04T11:54:06: %EASYPY-INFO: `-- VERIFY LEAF DEVICE(S) EXPECTED NEIGHBOR COUNT via NX-API CLI JS... PASSED
2024-02-04T11:54:06: %EASYPY-INFO: `-- 1_Verify BGP Neighbor Count PASSED
2024-02-04T11:54:06: %EASYPY-INFO: |-- 2_${device} IN @{device_list} PASSED
2024-02-04T11:54:06: %EASYPY-INFO: |-- 3_${device} = staging-leaf1 PASSED
2024-02-04T11:54:06: %EASYPY-INFO: |-- 4_Get BGP L2VPN Summary From Device PASSED
2024-02-04T11:54:06: %EASYPY-INFO: |-- 5_Nxapi Method Nxapi Cli PASSED
2024-02-04T11:54:06: %EASYPY-INFO: |-- 6_Log PASSED
2024-02-04T11:54:06: %EASYPY-INFO: |-- 7_ PASSED
2024-02-04T11:54:06: %EASYPY-INFO: |-- 8_Set Variable PASSED
2024-02-04T11:54:06: %EASYPY-INFO: |-- 9_Evaluate PASSED
2024-02-04T11:54:06: %EASYPY-INFO: |-- 10_Run Keyword If PASSED
2024-02-04T11:54:06: %EASYPY-INFO: |-- 11_Set Variable PASSED
2024-02-04T11:54:06: %EASYPY-INFO: `-- 12_Should Be Equal As Numbers PASSED
2024-02-04T11:54:06: %EASYPY-INFO: Done!
Once executed, close pyats.robot by clicking the small x (close button) to the right of the file tab:
Once completed, continue to the next section to get started with NetDevOps. In the next section you will take everything you have been developing and deploying to your staging (development) fabric to Git for source control then to your prod fabric through a devops process and a CI/CD pipleline.