Ansible module – automatically send out notification E-Mails to down BGP peers

Use case

I’ve been working with Ansible for some time now but wanted to see how much it takes to write a module so I made up the scenario where you have routers peering at different IX’s and your goal is to check periodically for all bgp sessions stuck in “CONNECT” state then do something about it. Either getting a summary E-mail with some more info and/or sending notifications to those peers.

After reading through couple of docs and testing I came up with some working code built as a python ansible module so vars could be handed over at the ansible playbook execution.

In networking, Ansible is not always used like in servers where you define tasks that will be pushed to the nodes of an inventory and all of it executed remotely. In some ways it loses some functionality but regarding how network devices work, the only generic way get things done remotely is sending CLI commands through the integrated SSH/CLI session handler called network_cli that uses paramiko, maybe over rest API or netconf if you are lucky.

Architecture

  • group_vars/
    • extreme.yml
  • library/
    • notify_peers_napalm
  • vars/
    • config.yml
  • inventory.yml
  • ansible_notify_peers_playbook.yml

group_vars/extreme.yml

That’s where we define the connection variables for the routers are defined. If you have other type of routers you can add such a file with its own vars in group_vars/. If you have devices of the same group “extreme” but a couple have different vars, you would create a “vars/” directory at the root of your project and add a .yml file named after the host entry in the inventory, add there the custom variables.

---
os: netiron
user: admin
passwd: password

library/notify_peers_napalm

This is the ansible module we wrote for this usecase. It isn’t complicated: it’s just a python script that imports the ansible modules:

from ansible.module_utils.basic import *

Then in the main function you have to handle the JSON variable mapping, and…. that’s it! The rest is your own python code, it’s all just a question of logic and understanding how Ansible is the most efficient and finding the best way to make it work with the task.

Note: Of course don’t forget that Ansible has been developed with the goal to deploy/provision/maintain services & gather infos on a lot of systems at the same time – it’s an automation tool. If you don’t automate anything with it, you might want to ask yourself if it’s the right tool for your needs, even if in networking the built-in CLI handlers come in so handy it’s difficult not to justify the usage.

vars/config.yml

Here we store variables that are global anyways but needed for our modules execution. In our example, we define the details about our own network/AS and our infos/credentials for our SMTP server and peeringdb.com account (AS contact information are not disclosed in guest mode).

---
myvars:
as: '65500'
noc_email: 'yournoc@domain.tld'
cpy_name: 'CompanyLTD'
peeringdb_url: 'https://www.peeringdb.com/search?q=65500'
peeringdb_usr: 'login'
peeringdb_pwd: 'password'
smtp_auth: 'true'
smtp_login: 'peering@domain.tld'
smtp_pwd: 'password'
smtp_host: 'mail.domain.tld'
smtp_sender: 'peering@domain.tld'
smtp_port: '587'

inventory.yml

The inventory file records the existing internet exchanges we want to get processed and related information to each IX. Here we define all the routers of type extreme (here 10.10.10.255 is the hostname string but we tricked with the IP because the network_cli will connect to it via SSH anyway). Under “vars” we specified the IX’s name and the IPv4 network it runs on.

---
all:
vars:
children:
extreme:
hosts:
10.10.10.255:
vars:
ix: 'INTERNET-EXCHANGE1'
network: '192.168.1.0'

ansible_notify_peers_playbook.yml

This is the playbook file where tasks are defined. What it does in our case is import the vars from the vars/config.yml, then it uses Napalm’s napalm_get_facts to get a standardized output equivalent to a CLI “show ip bgp summary” command, assign the output to a variable result and finally call our own notify_peers_napalm module to process that info:

---
- hosts: all
  connection: local
  gather_facts: no
  vars:
#    ansible_python_interpreter: "/usr/bin/env python"

  tasks:
    - name: Include only files matching config.yml
      include_vars:
        dir: vars
        files_matching: config.yml

    - name: collect bgp neighbors information
      napalm_get_facts:
        hostname={{ inventory_hostname }}
        username={{ user }}
        dev_os={{ os }}
        password={{ passwd }}
        filter='bgp_neighbors'
      register: result
 
    - name: process the show bgp in the module
      notify_peers_napalm:
        input: "{{ result }}"
        notify_peers: "False"
        notify_summary: "True"
        vars: "{{ myvars }}"
        ix: "{{ ix }}"
        router_ip: "{{ inventory_hostname  }}"
        network: "{{ network }}"

notify_peers_napalm module

Main Routine

#strip the raw string and split it into individual peering lines
rough_peer_list = objectify_json(input,network)

#transform the string list into a list of "peer" objects
enriched_peer_list = add_info_obj(rough_peer_list)

#generate a summary to your NOC and/or/but don't send an E-Mail to all down peers 
generate_email(enriched_peer_list,notify_peers,notify_summary,ix)

If you want more detail, you can find the project’s files and code in my github repo:

https://github.com/pirmins/notify_peers_napalm

Output sample

You’d receive a summary E-Mail at your NOC:

Following Peerings have been identified as in down state: 
(No E-Mails have been sent, to auto-send E-Mails to peers change notify_peers to true in vars/config.yml)

------------------------
------------------------

Peer E-Mail: noc@peeringispX.com
------------------------
Peering session with peer AS65000 down at IX1
------------------------
   AS       IPv4           State      Age   
  65501   192.168.1.176   CONN   34d23h4m

------------------------
------------------------

Peer E-Mail: noc@peeringispY.com
------------------------
Peering session with peer AS65000 down at IX1
------------------------
   AS       IPv4           State      Age   
  65502   192.168.1.236   CONN   12d2h1m

If you do change notify_peers to true in vars/config.yml, an E-Mail like this sample will be sent to each down peer:

Hello AS65501 NOC,
We are ISPXY AS65000 and are peering with you at IX1.
Unfortunately, our peering session is down:

   AS       IPv4           State      Age   
  65501   192.168.1.176   CONN   66d2h47m 

Could you please take a look at your side or notify us about any peering policy changes so we can take them into account.
Thank you,

Kind regards,
ISPXY NOC
E-Mail: noc@ispXY.com
PeeringDB: https://www.peeringdb.com/search?q=65000

Final note

This is a case study and as a whole is an interesting automation project to implement from A to Z (for the Z part, I will explain how I integrated this Ansible project into AWX to make it really interesting). While bigger ISPs already have centralized set of tools to administrate and maintain their peerings, this could still be of big interest to smaller ISPs.

Now, at beginning, I wrote this module for Extreme SLX OS solely. I had to write a module per router OS to extend router OS compatibility because I am parsing CLI output which can vary a lot from one vendor to another. Thanks to NAPALM’s “get_bgp_neighbor” getter instead of “slxos_command” & “show ip bgp neighbor summary“, its CLI abstraction and ouput normalization, I could rewrite the main parsing function and simplify the ansible playbook a lot.

Caution though, NAPALM doesn’t officialy support the full range of network OSs, “only” IOS, IOS-XR, JUNOS, EOS & NXOS, but there are community projects for other vendors (Extreme SLXOS beeing one of them) that are working & if lucky also maintained. When you think about it, you can make your code handle most of the industry vendors instead of just one, for the same amount of work involved. That’s pretty powerfull!

For more informations about NAPALM & its getters/setters compatibility table per OSes: https://napalm.readthedocs.io/en/latest/support/index.html

Community projects listed here: https://github.com/napalm-automation-community

My project: https://github.com/pirmins/notify_peers_napalm