**Or, Struggling with Python to get Working Automation Code**
{: .page__subtitle}
In the [last post](/automation/junipers-nornir-and-dynamic-inventories/), we talked about getting started with bulk configuration (mainly for Junipers) even if you are on a pre-1.6 Nautobot, which means you will not yet have access to Golden Configuration's fancy new remediation ability. Obviously we all want to upgrade as soon as we can to 1.6 and even 2.0+ for all the new features, but many organizations have Reasons for going more slowly. So in the meantime, we showed how nornir_pyez with nautobot as a dynamic nornir inventory can get you most of the way there for jinja config snippets. The only problem with it, however, is that the inventory only includes dcim.devices.
Fear not, intrepid small-wins automators! GraphQL is here to save the day. [Josh VanDeraa](https://josh-v.com/automation-inventory/) and [Ken Celenza](https://blog.networktocode.com/post/nautobot-ansible-variable-management-at-scale/) of NTC have discussed why it's a bad idea to include too much in the global inventory, and thus why they have limited `pynautobot_dictionary` to dcim.devices. In his post, however, Ken shows for Ansible how with embedded graphql queries, you can include extra variables like VLAN ID, interfaces and circuits on a host-by-host basis, and I wanted to see if we could do the same for task.host.data in nornir.
## Feeling impatient?
First off, let's help the expert python developers get in touch with their inner Prince Humperdinck, in case this is too basic:
[Skip to the end!](/automation/nautobot-graphql-and-dynamic-inventories/#embedding-graphql-into-python)
I myself loathe googling a pumpkin pie recipe and having to scroll through pages of someone's experiment with roasting their own pumpkins only to discover that nobody could tell the difference between that and canned pumpkin purée. Just give me the ingredients and steps already!
Also, you could use the TOC to the right to skip ahead to whichever section is most interesting. OK, with that done, let's get into it with some detail for those of us who are learning python to automate our networks, rather than the other way around.
## Digging in the Code: Python Libraries Edition
The first thing I did, of course, was ask my friend Google how to use graphql queries with our dynamic nornir inventory. I tried lots of keyword combinations but only ever came up with the Nautobot docs on general graphql usage, which I had scoured pretty well previously (see [my summary below](#writing-a-usable-query-with-graphiql)) . Next, I started digging in the code of Golden Config, because I figured they were already doing this with the SoTAgg queries that are specified with Jinja templates to generate intended configurations.
Not being a software developer, it was a revelation to discover that I could look into the site-packages in my python installation to see how those libraries I'd imported worked. To find your installed packages, use `python -m site` and look for `USER_SITE` near the end of the output (I cut out the sys.path output):
```shell
[12:50] ~ % python -m site
...
USER_BASE: '/Users/barrie/Library/Python/3.8' (exists)
USER_SITE: '/Users/barrie/Library/Python/3.8/lib/python/site-packages' (exists)
ENABLE_USER_SITE: True
````
Or, if you are using a virtual environment, it's in the lib directory of that. For example, in one of my repos where I'd installed a venv with its requirements.txt, I start typing `cd virt/lib` and hit tab twice to see the following suggestion:
```shell
cd virt/lib/python3.8/site-packages/
````
For the task at hand, I wanted to look where I'd installed the nautobot and nornir libraries, so kept going with `n` and ``.
```shell
[11:23] my-repo % cd virt/lib/python3.8/site-packages/nornir_nautobot/plugins/inventory
[11:35] inventory % ls -al
total 24
drwxr-xr-x 4 barrie staff 128 Dec 14 13:08 .
drwxr-xr-x 6 barrie staff 192 Sep 6 17:28 ..
-rw-r--r-- 1 barrie staff 33 Sep 6 17:28 __init__.py
-rw-r--r-- 1 barrie staff 5627 Sep 6 17:28 nautobot.py
````
Eventually after settling on that `nornir_nautobot` library, I continued to the `plugins` directory and viewed the `nautobot.py` file.
Turns out you can learn a lot by looking at code written by pros, but for a while you may be even more mystified until you hit upon the right thing to ask Stackoverflow (via Google). At this point in my python struggles, I was still pretty mystified by this particular file from the library, but it was somewhat interesting to see that the "pynautobot_dictionary" endpoint is set at the end of this file, and that the "name" of the host is set to device.name and that its "hostname" is actually set to the primary ipv4 address if it exists:
```python
if self.pynautobot_dict:
host["data"]["pynautobot_dictionary"] = dict(device)
# TODO: #3 Investigate Nornir compatability with dictionary like object
# Add Primary IP address, if found. Otherwise add hostname as the device name
host["hostname"] = (
str(ipaddress.IPv4Interface(device.primary_ip.address).ip) if device["primary_ip"] else device["name"]
)
host["name"] = device.name or str(device.id)
````
...but this still didn't really answer my question on how I was going to include other data via graphql, so I kept digging--it seemed like a good idea to next find out how Golden Config used the SoTAgg query to gather this data for intended configs.
## Digging in the Code: Python Source Code Edition
Because I'm running a Docker Compose install of nautobot, I wanted to head over to the [source code](https://github.com/nautobot/nautobot-app-golden-config) for the Golden Config app ([formerly called plugin](https://blog.networktocode.com/post/nautobot-app-rebranding/)) on github, but first I needed to know which version of the app I had running. To do this, I ran a `pip freeze` in my nautobot container---most `docker ps` fields and lines, as well as `pip freeze` lines, were deleted for brevity below). Note that the version of nautobot that this is installed with is 1.5.9---the app versions are numbered differently:
```shell
nautobot@nautobot:~$ docker ps
CONTAINER ID NAMES
abcdef21e337 mycompany-nautobot
nautobot@nautobot:~$ docker exec -it mycompany-nautobot bash
nautobot@abcdef21e337:~$ pip freeze
nautobot-golden-config==1.4.1
````
If you are running a VM-based nautobot, execute `pip freeze` on the command line, or in the virtual environment if you are using one. You may also be able to more easily view the app source here like we did with libraries.
{: .notice--info}
On Github, I navigated to the [nautobot-app-golden-config](https://github.com/nautobot/nautobot-app-golden-config) repo and then clicked on the `nautobot_golden_config` directory to see the app's python files. Because it defaults to the latest version, I needed to click on "develop" under Files at the upper left to switch branches/tags, click on the Tags tab, and select v.1.4.1:
In the `nornir_plays` directory, I scoured `config_intended.py` for clues as to how Golden Config was using the SOTAgg query to update the inventory but at the time I didn't notice this little tidbit on line 75:
```python
task.host.data.update(device_data)
````
I had gone straight to line 98 to pore over the config_intended method and in many ways it was educational, but my relative inexperience prevented me from seeing how to use this information without a job. I started googling again and found this great [post about creating jobs](https://blog.networktocode.com/post/writing-your-first-nautobot-job-part-01/) from NTC, and it was a very good explanation of Django ORM concepts, but it is Part 1 and stopped just short of actually creating jobs. In any case, I probably wouldn't have known what to do with line 75 of `config_intended.py` at this point either, so regardless it was time to ask the experts.
## Asking for Help from the Community
In this case, the [NTC slack](networktocode.slack.com) was the best place to ask, because the #nautobot and #nornir channels include lots of smart people both writing and using nautobot plugins. Hoping someone like a developer might see my question and be able to get me started, I asked about modifying the inventory. Josh replied with the links about "pynautobot_dictionary", and Ken suggested rendering the whole intended config in GC and post-processing it, but agreed that in 1.5 ths would not be possible. After a bit more discussion, he gave me a great skeleton for updating task.host.data with graphql output, and then I found some old code written by a co-worker that had embedded graphql and parsed the JSON output. I finally had the framework to
## Embedding GraphQL into Python
Once you have your graphql query, you can embed it into python. This can be in a separate file---which would require file handling, and for proof of concept I got pretty lazy---or just at the top of the python file after the imports and before the functions:
```python
import pynautobot
import json
from os import getenv
from nornir import InitNornir
myquery = '''
query ($device_id: ID!) {
device(id: $device_id) {
hostname: name
interfaces {
description
enabled
name
ip_addresses {
address
family
}
tagged_vlans {
vid
tenant {
name
}
}
untagged_vlan {
vid
tenant {
name
}
}
}
}
}
'''
def add_variable(task):
nb = pynautobot.api(
url="https://nautobot.mycompany.net",
token=getenv("NB_API_TOKEN"),
)
try:
gql = nb.graphql.query(
query=myquery,
variables={
"device_id": task.host["pynautobot_dictionary"]["id"]
})
except Exception as e:
print(str(e))
return None
else:
if gql.json.get('data'):
task.host.data.update(gql.json.get('data'))
return None
nr = InitNornir(
runner={"options": {"num_workers": 20}},
inventory={
"plugin": "NautobotInventory",
"options": {
"nautobot_url": "https://nautobot.mycompany.net",
"nautobot_token": getenv("NB_API_TOKEN"),
"filter_parameters": {"site": "NYC"},
"ssl_verify": True,
},
},
)
# Add graphql data to host
nr.run(task=add_variable)
````
So, we have to call nautobot twice; once as the nornir inventory so we have task.host, then again as pynautobot.api so we can initiate the graphql query for that host and use the results to update its nornir inventory. Ken Celenza of NTC suggested we could alternatively use the nautobot ORM for the latter, but because I was already using pynautobot and more familiar with it, I went with the API method.
Notice that the `$device_id` variable submitted with graphql in this case comes from "pynautobot_dictionary", because I needed to submit the ID of the current task.host. The only host data we have at this point is the standard nautobot inventory dictionary, so we can get the ID from there.
{: .notice}
In [Part 2](/automation/part2), we'll discuss how to actually use this with Jinja templates and nornir_pyez to update partial configs, such as for turning up a single new interface.