5 minute read

Or, where to start?

Ancient water clock (likely John Farey, Jr., 1791-1851)
Ancient water clock (likely John Farey, Jr., 1791-1851)

According to Wikipedia, automation is “a wide range of technologies that reduce human intervention in processes,” going back more than two millenia. Today we are no strangers to automation of the everyday sort – in fact, even ARPAnet embodied far more automation than Henry Ford could have fathomed at the height of the Model T.

When we talk of network automation, however, we’re talking about removing engineers from what they’ve spent 50-plus additional years assembling into the insanely complex spaghetti of interconnections that most can only conceive of as a cloud. Depending upon network age and acquisitions, there is often such a mishmash of technologies, vendors and data storage (including in some people’s heads) that the task of removing any sort of intervention by engineers can feel insurmountable.

Most networks are not starting from scratch, after all, and have the practical consideration of integrating those existing systems with any new automation deployed, without causing new issues. Even greenfield networks have to consider how to choose everything: equipment vendors, real estate, the myriad OSS, IT, CRM, and other platforms–how it will all interoperate, assuming it does, and not cost an arm and a leg?

Thinking of the problem as insurmountable, however, is the real problem, because of course to realize the benefits of automation, we all have to start somewhere.

So keep in mind, when thinking about network automation, that it does not have to be all or nothing. Working with what you’ve got now benefits everyone, and gives you space to move forward only when all stakeholders are comfortable with each successive state of automation. To this end, it can be helpful to identify stages of automation readiness and what can be done at each one, and I think I’ve found four so far (maybe even five).

Before we dive into them, however, let’s talk about Golden Configuration. A Google search yields any number of different companies who have defined this in hopes of selling their own version, but it selects IBM’s as the best and I, too, find it to be the most general and succinct:

“A ‘golden’ configuration is a configuration version used in compliance management as an ideal configuration against which configurations from similar devices can be compared.”

Although they did use the word “configuration” too many times and it’s starting to sound weird – I’m going to call it GC from here on out if I can. In general, it can be a good way to start automating, since it doesn’t have to involve any configuration pushes, yet can help keep configuration drift reined in.

I’ve recently used Nautobot’s GC plugin (app) for this–it allows you to decide on policy, generate an intended configuration for each device with templates, and compare it to actual configuration backups and see where it differs. Network to Code provides a demo nautobot instance to try out features with some sample data before jumping fully into installation and setup, so I’ll be using this for any examples and screenshots going forward.

Nautobot: GC compliance by feature

Through the UI, you can specify configuration feature categories such as bgp, dns, snmp, etc. and see at a glance whether they are in compliance or not. You can also drill down into these features and see exactly what has changed:

Missing and Extra configuration lines for interfaces feature

So, given some sort of framework like this, how does a netops team begin to automate what may be a patchwork of vendors, software versions, and configuration policies inherited from long gone engineers or other companies? Start backing up configs to a repositorynote, and pick the stage you can work with – I’ll have detailed posts on each one in the near future.

Stage 0 - Diff Backups with Baseline Configs

I’m not sure this counts as a true automation stage, so it’s getting numbered as zero, and we can still say there are only four stages if we wish. But this will give the organization more information than it had before, and require very little of what we think of as the hard work of automation to get started, so it’s worth mentioning.

Once we get automated backups going to a repository, we can create a copy of that repository and keep it (mostly) static. That second copy is the baseline, and can be used with configuration compliance to at least give you an idea of what’s changing from that point in time on, or until you make a new copy.

Edit - 10/11/2023: It turns out there is another great way to start out automating with the source of truth, and that’s using the SSOT as the nornir inventory–no baseline Golden Config required. More on that in an upcoming post!

Stage 1 - Diff Backups with Intended Configs

The first real stage! Compare backups with intended configurations to ensure policy compliance. This of course requires creating a policy in the first place, along with templates at some degree of sophistication to generate intended configs, and a device inventory like netbox or nautobot.

Stage 2 - Push intended configs to devices

Once the NetOps org has confidence in the intended configurations generated, they can decide when to push them directly to the devices, rather than individuals configuring devices via the CLI or a proprietary management system.

Stage 3 - Push intended configs automatically

This can be done on a schedule, or with job triggers or web hooks that detect when the policy or intended has changed.

Stage 4 - Update the inventory automatically from other business processes

This can be triggered by a new service or customer provisioned from a CRM, ERP or other business-focused platform, and this stage can actually precede any of the other stages, or be the final step toward full automation. Once the entire provisioning process from sales to turnup can be done this way, many sources of error and barriers to growth are eliminated.

note: If you don’t know what I’m talking about above regarding repositories, make sure you have a github organization and we’ll get there soon enough. (Edit - 10/11/2023: Oh look! We’re there.)