Table of Contents
Using google TextFSM parser in Nornir automation scripts helps us get structured output when sending show monitoring commands to network devices from which any information can be easily extracted.
In this section we show how to use textfsm in nornir Netmiko and Scrapli plugins.
In the next section, we will introduce the Genie parser, which belongs to the Cisco Company.
TextFSM Parser and Nornir
What is TextFSM Parser?
As you know, in Nornir Naplm plugin, the output of show monitoring commands are structured in JSON format, which allows us to easily extract the information using lists and dictionaries in the Python programming language.
But in Scrapli and Netmiko plugins, to extract specific information from the output of show monitoring commands, a regular expression must be used to parse the text to get the required information.
In this section, we’ll use the Google Textfsm parser, which will help us extract information from the output of show monitoring commands without using regular expressions.
In order to use textfsm, we need to prepare a template file that specifies the required and the format of the information.
Network to Code is an automation focused company that has already prepared many templates for the most important monitoring commands from many different vendors and operating systems. Templates prepared by network to code company is called “ntc templates”.
The list of ntc templates can be find in github open source community.
As you can see the most important vendors and operating systems are in the list of ntc templates.
It includes the most important monitoring commands of each device.
Comparing to cisco genie which will be discussed in the next section, more vendors are supported but the list of supported commands are limited.
To check inside of a template as an example, I choose cisco ios “show ip interface brief” command.
The information extracted from this template is the interface name, the interface IP address, and finally the physical and line protocol status of the interface.
The type of information and the format of the output are also specified in the template.
Value INTF (\S+)
Value IPADDR (\S+)
Value STATUS (up|down|administratively down)
Value PROTO (up|down)
Start
^${INTF}\s+${IPADDR}\s+\w+\s+\w+\s+${STATUS}\s+${PROTO} -> Record
# Capture time-stamp if vty line has command time-stamping turned on
^Load\s+for\s+
^Time\s+source\s+isNornir netmiko and nornir scrapli can use textfsm to parse the output of show monitoring commands to easily extract any information.
TextFSM with Nornir Scrapli Plugin
We will use textfsm with both scrapli and netmiko since the usage a little different.
In the first script we use scrapli to send “show ip interface brief” command to network devices and use textfsm to parse the output of this command in order to extract IP address and status of each interface.
from nornir import InitNornir from nornir_scrapli.tasks import send_command from rich import print as rprint nr = InitNornir(config_file="config.yaml") def nornir_scrapli_textfsm_example(task): result=task.run(task=send_command, command="show ip interface brief") interfaces = result.scrapli_response.textfsm_parse_output() rprint(interfaces) # rprint(type(interfaces)) # for interface in interfaces: # rprint(interface) # rprint(type(interface)) # rprint("interface", interface["intf"], "with IP address", interface["ipaddr"] , "is physically ", interface["status"], "and line protocol is", interface["proto"]) results=nr.run(task=nornir_scrapli_textfsm_example)
In this script I use rprint from rich library only to display the structured output in a nicer format, otherwise there is no difference to using print or rprint.
To use textfsm in nornir scrapli plugin, we have stored the result of “show ip interface brief” command in “result” variable.
Any “result” output will be parsed into structured output using “result.scrapli_response.textfsm_parse_output()” command if the command is supported.
The structured output is stored in “interfaces” variable and it is finally printed.
The other section of the script are commented and will be discussed after that.
Let’s run the scrip and see the result.
majid@devnet:~/devnet/pyhton_nornir/2023/8.textfsm_parser$ python3 8.1.nornir_scrapli_textfsm_example.py
[
{'intf': 'GigabitEthernet1', 'ipaddr': '192.168.2.91', 'status': 'up', 'proto': 'up'},
{'intf': 'GigabitEthernet2', 'ipaddr': 'unassigned', 'status': 'administratively down', 'proto': 'down'},
{'intf': 'GigabitEthernet3', 'ipaddr': 'unassigned', 'status': 'administratively down', 'proto': 'down'}
]
As you can see the result of the script is structured and can be easily processed. The list of interfaces, IP address of each interface and physical and line protocol of each interface are displayed in the output as it is defined in the template.
The advantage of structured output is that it is in the format of list and dictionary and can be easily processed to extract more specific information, like IP addresses of interfaces.
Process list and dictionary in python to extract more specific information
How can I understand that what is the format of the output. Is it a dictionary or a list?
To process a list, we need a loop to access any component of the list.
To process a dictionary, we give the “key” and receive the “value”.
I use the “type” keyword to find type of any variable.
rprint(type(interfaces))
Let’s run the script again to see what type of variable the “interfaces” variable is.
majid@devnet:~/devnet/pyhton_nornir/2023/8.textfsm_parser$ python3 8.1.nornir_scrapli_textfsm_example.py <class 'list'>
The result shows that “interfaces” variable is in the format of list.
Therefore in the next step, I use a loop to access each component of the list.
In the first step, we just print each element of the list which has the same output.
for interface in interfaces: rprint(interface)
majid@devnet:~/devnet/pyhton_nornir/2023/8.textfsm_parser$ python3 8.1.nornir_scrapli_textfsm_example.py
{'intf': 'GigabitEthernet1', 'ipaddr': '192.168.2.91', 'status': 'up', 'proto': 'up'}
{'intf': 'GigabitEthernet2', 'ipaddr': 'unassigned', 'status': 'administratively down', 'proto': 'down'}
{'intf': 'GigabitEthernet3', 'ipaddr': 'unassigned', 'status': 'administratively down', 'proto': 'down'}To see the type of each component of the list, again I use the “type” keyword. Each element of the list is located in “interface” variable in the loop. Therefore “type(interface)”, shows the type of each element of the list.
for interface in interfaces: rprint(type(interface))
Let’s run the script again to see what type of variable the “interface” variable is.
majid@devnet:~/devnet/pyhton_nornir/2023/8.textfsm_parser$ python3 8.1.nornir_scrapli_textfsm_example.py <class 'dict'> <class 'dict'> <class 'dict'>
The output shows that each element of the list is a dictionary. And therefore “intf”, “ipaddr”, “status” and “proto” are the keys of the dictionary.
By specifying any of these keys, we can access its value.
In the last line of the “for loop”, I access each element of the output separately by parsing the dictionary’s key and printing the result in customized format.
let’s run the script to print each value of the structured output in a customized format.
for interface in interfaces: rprint("interface", interface["intf"], "with IP address", interface["ipaddr"] , "is physically ", interface["status"], "and line protocol is", interface["proto"])
majid@devnet:~/devnet/pyhton_nornir/2023/8.textfsm_parser$ python3 8.1.nornir_scrapli_textfsm_example.py
interface GigabitEthernet1 with IP address 192.168.2.91 is physically up and line protocol is up
interface GigabitEthernet2 with IP address unassigned is physically administratively down and line protocol is
down
interface GigabitEthernet3 with IP address unassigned is physically administratively down and line protocol is
down
TextFSM with Nornir Nemiko Plugin
This is the second script that uses textfsm in the Nornir Netmiko plugin to access specific information of the structured output of the show monitoring command.
from nornir import InitNornir from nornir_netmiko.tasks import netmiko_send_command from rich import print as rprint nr = InitNornir(config_file="config.yaml") def nornir_netmiko_textfsm_example(task): result=task.run(task=netmiko_send_command, command_string="show interfaces", use_textfsm=True) interfaces = result.result rprint(interfaces) # rprint(type(interfaces)) # for interface in interfaces: # rprint(interface) # rprint(type(interface)) # rprint("interface", interface["interface"], "with IP address", interface["ip_address"], "is physically ", interface["link_status"], "and line protocol is", interface["protocol_status"]) results=nr.run(task=nornir_netmiko_textfsm_example)
In this script we send “show interfaces” command , which gives us more information of the interfaces compared to the “show ip Interface brief” command.
In the Nornir Netmiko plugin, unlike Nornir Scrapli, textfsm is activated as an argument when we run a task.
Here we store the result of the running task in the “result” variable.
Then we run “result.result” to store the structured output in the “interfaces” variable.
Then we repeat exactly what we did with textfsm in the nornir scrapli plugin to access specific information of the structured output.
In each step we check whether the output is a list or a dictionary. If it’s a list, we’ll access the elements of the list via a loop, and if it’s a dictionary, we’ll access each value by specifying the key.
First of all let’s run the script and see the complete structured output of interfaces variable.
majid@devnet:~/devnet/pyhton_nornir/2023/8.textfsm_parser$ python3 8.2.nornir_netmiko_textfsm_example.py [ { 'interface': 'GigabitEthernet1', 'link_status': 'up', 'protocol_status': 'up', 'hardware_type': 'CSR vNIC', 'address': '000c.2971.8b31', 'bia': '000c.2971.8b31', 'description': '', 'ip_address': '192.168.2.91/24', 'mtu': '1500', 'duplex': 'Full Duplex', 'speed': '1000Mbps', 'media_type': 'Virtual', 'bandwidth': '1000000 Kbit', 'delay': '10 usec', 'encapsulation': 'ARPA', 'last_input': '00:00:00', 'last_output': '00:00:00', 'last_output_hang': 'never', 'queue_strategy': 'fifo', 'input_rate': '16000', 'output_rate': '0', 'input_packets': '19939', 'output_packets': '710', 'input_errors': '0', 'crc': '0', 'abort': '', 'output_errors': '0' }, { 'interface': 'GigabitEthernet2', 'link_status': 'administratively down', ... }, { 'interface': 'GigabitEthernet3', 'link_status': 'administratively down', ... } ]
Process list and dictionary in python to extract more specific information
In the first step we check if the format of the “interfaces” is a list or a dictionary through giving “type” keyword.
rprint(type(interfaces))
majid@devnet:~/devnet/pyhton_nornir/2023/8.textfsm_parser$ python3 8.2.nornir_netmiko_textfsm_example.py <class 'list'>
The result of the script shows that “interfaces” variable is a list.
Then we check the type of each element in the list with the “type(interface)” keyword.
for interface in interfaces: rprint(type(interface))
majid@devnet:~/devnet/pyhton_nornir/2023/8.textfsm_parser$ python3 8.2.nornir_netmiko_textfsm_example.py <class 'dict'> <class 'dict'> <class 'dict'>
As we expect, each element of the list a dictionary and we can access the values of the interface with giving the keys, “interface”, “link status”, “protocol_status”, “hardware_type” and so on.
In the last line of the “for loop”, I access the interface name, IP address, and physical and protocol status by specifying the appropriate keys.
for interface in interfaces: rprint("interface", interface["interface"], "with IP address", interface["ip_address"], "is physically ", interface["link_status"], "and line protocol is", interface["protocol_status"])
majid@devnet:~/devnet/pyhton_nornir/2023/8.textfsm_parser$ python3 8.2.nornir_netmiko_textfsm_example.py
interface GigabitEthernet1 with IP address 192.168.2.91/24 is physically up and line protocol is up
interface GigabitEthernet2 with IP address is physically administratively down and line protocol is down
interface GigabitEthernet3 with IP address is physically administratively down and line protocol is downTextFSM scripts can be download from this link.