Backup your Network Device Configurations Before Someone Breaks Them!

Do you guys remember the movie Minority Report? In this movie, there are three psychics called precogs that are part of “Precrime”, a specialist police department that could apprehend criminals based on foreknowledge provided by the precogs. I have to admit that I like the movie because it makes me think whether this is something that would be accepted in our society in the present day. But also makes me wonder if we could do something similar with our network devices, and catch the bad guy before something nasty happens on your network. In this case, we don’t want to put anyone in jail, we just want to be sure that we have the tools to identify, take action and revert any changes.

Using SolarWinds® Network Configuration Manager (NCM) and Cisco Embedded Event Manager (EEM), today we will configure your SolarWinds platform to take a backup of the running config from your network devices when someone enters in configuration mode. This way you will be sure that there is a fresh copy of the device configuration stored in SolarWinds before someone breaks the configurations of your devices.

Time to configure EEM
The first thing we have to do is to configure your devices to notify SolarWinds when someone gets into configuration mode. This is something that can be done using Cisco EEM technology, let me show you how to do it.

The configuration of the Cisco device is easy, just run these commands in config mode:

[code]event manager applet EEM
event cli pattern "configure terminal" sync no skip no
action 1.0 cli command "enable"
action 2.0 cli command "sh users | incl [0-9]+\.[0-9]+\.[0-9]+\.[0-9]+"
action 3.0 syslog priority notifications msg "Entered config mode. Users: $_cli_result"[/code]

The first command defines the name of the EEM applet, any name would do.

The second command defines when this applet is triggered, and this happens when someone executes “configure terminal”, this includes any aliases such as “config t”, “conf t” …

Finally when the event is detected, the device will trigger action 1.0 to allow show commands, then action 2.0 to show which users are connected remotely to the device, and finally action 3.0 that will send a Syslog message including the results of the show command in action 2.0,  together with a Notification (level 5) severity level. Feel free to modify the message content or the severity of the message, however, any change will have to be reflected on the alert that we will create in SolarWinds as outlined below.

 

Now we have to make sure your SolarWinds server receives Syslogs from this device. If you haven’t configured your device yet, you can configure it with the following commands:

[code]Logging x.x.x.x
Logging trap 5[/code]

Where x.x.x.x is the IP address of your SolarWinds server.

Time to Configure SolarWinds
We could simply use the existing SolarWinds Real-Time Change Detection (RTCD) feature within NCM, but this will capture the post config change, whereas here we are making sure we capture the pre-change config.

The method we want to deploy here is creating a common alert in SolarWinds that detects when the Syslog message is received and then goes back to the device and gets a fresh copy of the running config. Please follow these steps to configure the alert:

Alert Properties
The main configurations here is the evaluation frequency. This value defines how often the SolarWinds alerting engine goes back to the database and checks if there is any Syslog message matching the trigger conditions. We recommend to leave the default value of one minute, however, this means that it might take up to one minute since the user enters into config mode and the backup is taken. If you want to minimize this time, I would recommend reducing the evaluation frequency to 15 seconds, although this may increase the SQL server load (not recommended for busy SQL servers).

Trigger Condition
As there is no way to create an alert for Syslog messages using the default options available using the graphical interface, we will use a SQL alert:
[code]Select Nodes.caption, nodes.nodeid from nodes
Inner join syslog on nodes.nodeid=syslog.nodeid
Where syslog.message like ‘%Entered config mode%’
And datediff(minute,syslog.datetime,getdate())[/code]

Important things here:

• The message on ‘Where syslog.message like ‘Entered config mode%’ has to match the message received on the syslog message.
• We check all the syslog messages received in the past 10 minutes. In theory, matching the alert evaluation frequency (one minute) should be enough, however just to be sure everything works smoothly I would recommend leaving some buffer time.

Reset Condition and Time of Day
Skip these two steps, the default configuration works for us.

Trigger Actions
Add a new action “Execute NCM action” and select “backup running config”. As we have to tell SolarWinds which device we have to take the backup from, enable the option “Enter the IP address of the NCM node where the NCM action will execute” and type the variable the will give us the IP address of the device that triggered the alert “${N=SwisEntity;M=IP_Address}”.

Reset Action
No action works for us.

Summary
Review if everything configured is okay and finish the alert.

Testing Time
Configuring EEM on your device and configuring the SolarWinds alert is all we need to do in order to make sure you have a copy of the running config before someone starts configuring your network devices. Now is time for you guys to go ahead and try these steps on your network.

NOTE: I left a copy of the alert definition to trigger the device backup.