Nautobot, GraphQL and a Dynamic Inventory, Part 1
Or, Struggling with Python to get Working Automation Code
In the last post, we talked about getting started with bulk configuration (mainly for Junipers) even if you are on a pre-1.6 Nautobot, which means you will not yet have access to Golden Configuration’s fancy new remediation ability. Obviously we all want to upgrade as soon as we can to 1.6 and even 2.0+ for all the new features, but many organizations have Reasons for going more slowly. So in the meantime, we showed how nornir_pyez with nautobot as a dynamic nornir inventory can get you most of the way there for jinja config snippets. The only problem with it, however, is that the inventory only includes dcim.devices.

Fear not, intrepid small-wins automators! GraphQL is here to save the day. Josh VanDeraa and Ken Celenza of NTC have discussed why it’s a bad idea to include too much in the global inventory, and thus why they have limited pynautobot_dictionary
to dcim.devices. In his post, however, Ken shows for Ansible how with embedded graphql queries, you can include extra variables like VLAN ID, interfaces and circuits on a host-by-host basis, and I wanted to see if we could do the same for task.host.data in nornir.
Feeling impatient?
What follows is a pretty deep dive into how an old-school network engineer might go about stumbling through python and jinja2 development as a de facto crash course in network automation. So, first off, let’s help the expert python developer get in touch with their inner Prince Humperdinck, in case this is all a little too basic: Skip to the end!
I myself loathe googling a pumpkin pie recipe and having to scroll through pages of someone’s experiment with roasting their own pumpkins only to discover that nobody could tell the difference between that and canned pumpkin purée. Just give me the ingredients and steps already!
You could also just use the TOC to the right to skip ahead to whichever section is most interesting, but Princess Bride references are more fun, aren’t they? With that done, let’s get into it with some detail for those of us who are learning python to automate our networks, rather than the other way around.
Digging in the Code: Python Libraries Edition
The first thing I did, of course, was ask my friend Google how to use graphql queries with our dynamic nornir inventory. I tried lots of keyword combinations but only ever came up with the Nautobot docs on general graphql usage, which I had scoured pretty well previously (see my summary in Part 2). I thought perhaps there might be something about this in the nornir_nautobot
library, so decided to go digging in that code next.
Finding python/site-packages
Not being a software developer, it was a revelation to discover that I could look into the site-packages
of my python installation to see how those libraries I’d imported worked. To find your installed packages, use python -m site
and look for USER_SITE
near the end of the output (I cut out the sys.path output):
[12:50] ~ % python -m site
...
USER_BASE: '/Users/barrie/Library/Python/3.8' (exists)
USER_SITE: '/Users/barrie/Library/Python/3.8/lib/python/site-packages' (exists)
ENABLE_USER_SITE: True
Or, if you are using a virtual environment, it’s in the lib directory of that. For example, in one of my repos where I’d installed a venv called virt
with its requirements.txt, I start typing cd virt/lib
and hit tab twice to see the following suggestion:
cd virt/lib/python3.8/site-packages/
Reading Library Source Code
For the task at hand, I wanted to look where I’d installed libraries related to nautobot and nornir, so kept going with n
and <tab>
.
[11:23] my-repo % cd virt/lib/python3.8/site-packages/nornir_nautobot/plugins/inventory
[11:35] inventory % ls -al
total 24
drwxr-xr-x 4 barrie staff 128 Dec 14 13:08 .
drwxr-xr-x 6 barrie staff 192 Sep 6 17:28 ..
-rw-r--r-- 1 barrie staff 33 Sep 6 17:28 __init__.py
-rw-r--r-- 1 barrie staff 5627 Sep 6 17:28 nautobot.py
Eventually after settling on that nornir_nautobot
library, I continued to the plugins
directory and viewed the nautobot.py
file.
Turns out you can learn a lot by looking at code written by pros, but for a while you may be even more mystified until you hit upon the right thing to ask Stackoverflow (via Google). At this point in my python struggles, I was still pretty mystified by this particular file from the library, but it was somewhat enlightening to see how task.host.data is set to “pynautobot_dictionary”:
if self.pynautobot_dict:
host["data"]["pynautobot_dictionary"] = dict(device)
# TODO: #3 Investigate Nornir compatability with dictionary like object
# Add Primary IP address, if found. Otherwise add hostname as the device name
host["hostname"] = (
str(ipaddress.IPv4Interface(device.primary_ip.address).ip) if device["primary_ip"] else device["name"]
)
host["name"] = device.name or str(device.id)
This still didn’t really answer my question on how I was going to include other data via graphql, so I kept digging. I moved on to the source code of the Golden Config app, because I figured they were already doing this with the SoTAgg queries that are specified with Jinja templates to generate intended configurations.
Digging in the Code: Python Source Code Edition
Because I’m running a Docker Compose install of nautobot, I wanted to head over to GitHub to find the source code for the Golden Config app (formerly plugin), but first I needed to know which version of the app I had running. To do this, I ran a pip freeze
in my nautobot container—most docker ps
fields and lines, as well as pip freeze
lines, were deleted for brevity below). Note that the version of nautobot that this is installed with is 1.5.9—the app versions are numbered differently:
nautobot@nautobot:~$ docker ps
CONTAINER ID NAMES
abcdef21e337 mycompany-nautobot
nautobot@nautobot:~$ docker exec -it mycompany-nautobot bash
nautobot@abcdef21e337:~$ pip freeze
nautobot-golden-config==1.4.1
If you are running a VM-based nautobot, execute pip freeze
on the command line, or in the virtual environment if you are using one. You may also be able to more easily view the app source here like we did with libraries.
With this information, I clicked on the nautobot_golden_config
directory link in the repo to see the app’s python files. Because it defaults to the latest version, I needed to click on “develop” under Files at the upper left to switch branches/tags, click on the “Tags” tab, and select v.1.4.1:

In the nornir_plays
directory, I scoured config_intended.py
for clues as to how Golden Config was using the SoTAgg query to update the inventory but at the time I didn’t notice this little tidbit on line 75:
task.host.data.update(device_data)
I had gone straight to line 98 to pore over the config_intended method and in many ways it was educational, but my relative inexperience prevented me from seeing how to use it without a job. I started googling again and found this great post about creating jobs from NTC, and it was a very good explanation of Django ORM concepts, but it is Part 1 and stopped just short of actually creating jobs. In any case, I probably wouldn’t have known what to do with line 75 of config_intended.py
at this point either, so regardless it was time to ask the experts.
Asking for Help from the Community
In this case, the NTC slack was the best place to ask, because the #nautobot
channel includes lots of smart people both writing and using nautobot plugins. Hoping someone like a developer might see my question and be able to get me started, I asked about modifying the inventory. Josh VanDeraa replied with his and Ken Celenza’s posts about why “pynautobot_dictionary” is limited in scope, and Ken suggested rendering the whole intended config in Golden Config and post-processing it, but agreed that in 1.5 remediation would not be possible.
I had also asked about Part 2 of the jobs post, and Stephen Corry of NTC replied that it was forthcoming—since it’s taken me a while to write this post, it’s now out, and I just need to find time to read it: Writing Your First Nautobot Job, Part 2
After a bit more discussion about what I was trying to do, Ken gave me a skeleton for updating task.host.data with graphql output, and then I found some old python code written by a co-worker that had embedded a graphql query and parsed the JSON output. I finally had the framework to put it all together!
Embedding GraphQL into Python
Assuming you have a working graphql query (again, see Part 2 for help with that), you can embed it into python. This can be in a separate file—which would require file handling, and for proof of concept I got pretty lazy—or using triple single-quotes at the top of the python file after the imports and before the functions:
import pynautobot
import json
from os import getenv
from nornir import InitNornir
myquery = '''
query ($device_id: ID!) {
device(id: $device_id) {
hostname: name
interfaces {
description
enabled
name
ip_addresses {
address
family
}
tagged_vlans {
vid
tenant {
name
}
}
untagged_vlan {
vid
tenant {
name
}
}
}
}
}
'''
def add_variable(task):
nb = pynautobot.api(
url="https://nautobot.mycompany.net",
token=getenv("NB_API_TOKEN"),
)
try:
gql = nb.graphql.query(
query=myquery,
variables={
"device_id": task.host["pynautobot_dictionary"]["id"]
})
except Exception as e:
print(str(e))
return None
else:
if gql.json.get('data'):
task.host.data.update(gql.json.get('data'))
return None
nr = InitNornir(
runner={"options": {"num_workers": 20}},
inventory={
"plugin": "NautobotInventory",
"options": {
"nautobot_url": "https://nautobot.mycompany.net",
"nautobot_token": getenv("NB_API_TOKEN"),
"filter_parameters": {"site": "NYC"},
"ssl_verify": True,
},
},
)
# Add graphql data to host
nr.run(task=add_variable)
So, we have to point to nautobot twice; once as the nornir inventory so we have task.host, then again as pynautobot.api so we can initiate the graphql query for that host and use the results to update its task.host.data. During our previous slack conversation, Ken Celenza of NTC had suggested we could alternatively use the nautobot ORM for the latter, but because I was already using pynautobot and more familiar with it, I went with the API method.
Notice that the $device_id
variable submitted with graphql in this case comes from “pynautobot_dictionary”, because that’s the only data we have for task.host at this point in the process.
In Part 2, I cover how to actually use this with Jinja templates and nornir_pyez to update partial configs, such as for turning up a single new interface. See you there!