Network Device Configuration Management - Oxidized

In this lab you will install Oxidized and integrate it with LibreNMS.

Requirements:

Completed network setup of https://ws.learn.ac.lk/wiki/Cnbp2022/Agenda/LabSetup (Download WKFiles files from this link and replace WKFiles with it your setup ​https://docs.learn.ac.lk/index.php/s/0kmjbRMJM2BIlYS)

Ubuntu 20.04 Server with LibreNMS installed (Use this provided VM and replace Monitoring_Server with it your setup ​https://docs.learn.ac.lk/index.php/s/WmuORn2bAnU7yNY)

Gmail account with 2-Step Verification On.

Oxidized Installation

Since most of the tasks in this lab require you to be "root", the first thing you should do is to connect to your server and start a root shell.

sudo su
apt update
apt-get install ruby ruby-dev libsqlite3-dev libssl-dev pkg-config cmake libssh2-1-dev libicu-dev zlib1g-dev g++
sysctl -w net.ipv6.conf.all.disable_ipv6=1
sysctl -w net.ipv6.conf.default.disable_ipv6=1
sysctl -w net.ipv6.conf.lo.disable_ipv6=1
gem install oxidized
gem install oxidized-script oxidized-web 

LibreNMS Integration

Create Token as follows

Login to LibreNMS web GUI (10.1.5.10) using WinXP_INSIDE. Go to Settings -> API Settings

Click on 'Create API access token'.

Click 'Create API Token'.

Login to CLI as libreNMS user and continue.

sudo su - librenms
lnms config:set oxidized.enabled true
lnms config:set oxidized.url http://127.0.0.1:8888
lnms config:set oxidized.features.versioning true
lnms config:set oxidized.group_support true
lnms config:set oxidized.default_group f

Oxidized Configuration

Create the oxidized user.

useradd -s /bin/bash -m oxidized
sudo su - oxidized
Oxidised
nano .config/oxidized/config

Change the configuration file by adding / updating the following information. Replace oxidizedfor (gmail), cisco (telnet/ssh username), 123 (telnet/ssh password), 09db454fb0ec6927c777e214c2f82d60 (LibreNMS token value) with your corresponding values. Interval value (in seconds) is the auto refresh interval (you can change that one).

---
username: username
password: password
model: ios
resolve_dns: false
interval: 300
use_syslog: false
debug: false
threads: 30
timeout: 20
retries: 3
prompt: !ruby/regexp /^([\w.@-]+[#>]\s?)$/
rest: 127.0.0.1:8888
next_adds_job: false
vars:
  enable: '123'
groups: {}
models: {}
pid: "/home/oxidized/.config/oxidized/pid"
crash:
  directory: "/home/oxidized/.config/oxidized/crashes"
  hostnames: false
stats:
  history_size: 10
input:
  default: telnet, ssh
  debug: false
  ssh:
    secure: false
  ftp:
    passive: true
  utf8_encoded: true
output:
  default: git
  git:
    user: Oxidized
    email: oxidizedfor@gmail.com
    repo: "~/.config/oxidized/oxidized.git"
source:
  default: http
  debug: true
  http:
    url: http://127.0.0.1/api/v0/oxidized
    map:
      name: hostname
      model: os
      group: group
    headers:
      X-Auth-Token: '09db454fb0ec6927c777e214c2f82d60'
groups:
  f:
    username: 'cisco'
    password: '123'
hooks:
  email_output:
    type: exec
    events: [post_store]
    cmd: '/home/oxidized/.config/oxidized/oxidized-report-git-commits -s "Oxidized updates for ${OX_NODE_NAME}" -r oxidizedfor@gmail.com'
    async: true
    timeout: 120

Setup Oxidized Service

Run the following commands as root user.

cp extra/oxidized.service /etc/systemd/system
mkdir /run/oxidized
chown oxidized:oxidized /run/oxidized
systemctl enable oxidized.service

Start oxidised service

systemctl start oxidized

Restart oxidised service

systemctl restart oxidized

View status of oxidised service

systemctl status oxidized

Login to LibreNMS GUI by browsing 10.1.5.10 in the web using WinXP_INSIDE.

Oxidized can be accessed as in the following.

A device configuration can be accessed by clicking the icon (see the below red arrow).

In the resulting screen, you can view Show version and Show diff buttons if there are configuration changes/ changing history, after Oxidized has connected with the devices. Try to get that screen by changing a configuration of a connected device.

Click Show diff. Below Hide line numbers a minus (-) mark in front of the command says that particular command is reverted (red). And a plus (+) mark in front of the command says the reverted command is replaced by it (green).

Postfix Installation

Postfix will be installed to have an email notification of configuration changes.

apt install postfix

Select Internet Site to enable Postfix to sent and receive mails and press Enter to proceed.

Put a suitable domain name for System mail name and proceed.

Gmail Configuration

Go to ​https://myaccount.google.com/ and sign into your Gmail account.

Select Security.

Under "Signing in to Google," select App Passwords.

Generate app password by selecting custom option as follows and note down the password.

Postfix Configuration

Here Postfix will be configured to send email to your Gmail account. First make sure that you are using root account.

vim /etc/postfix/main.cf

Go through file carefully and add / update the necessary lines as below. Don't fully remove any default line. Some default lines are also depicted in the below configuration to aid positioning lines in the correct place.

smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_tls_security_level=may

smtp_tls_CApath=/etc/ssl/certs
smtp_tls_security_level=may
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt

alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
relayhost = [smtp.gmail.com]:587
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous

vim /etc/postfix/sasl_passwd

Add the following content to /etc/postfix/sasl_passwd while changing oxidizedfor and gecqeehdvtrrsed5 (app password).

[smtp.gmail.com]:587 oxidizedfor@gmail.com:gecqeehdvtrrsed5

Close the above file and continue.

chown root:root /etc/postfix/sasl_passwd
chmod 600 /etc/postfix/sasl_passwd
postmap /etc/postfix/sasl_passwd
systemctl restart postfix

Check email sending. Replace oxidizedfor accordingly.

echo "Test Postfix Gmail SMTP Relay" | mail -s "Postfix Gmail SMTP Relay" oxidizedfor@gmail.com

An email will be sent to the Gmail account after a saved configuration change, when Oxidized get refreshed.

Such email content will be in a similar format as follows. There the minus (-) mark in front of the command says that particular command is reverted. And plus (+) mark in front of the command says the reverted command is replaced by it.