Alerting and Notification on Icinga2

Like in LibreNMS, we can setup multiple alerting mechanisms in Icinga2, and since Icinga Stack is a more advanced software, any type of custom notification system can be associated. In this tutorial, we will only focus on setting up email, slack and telegram alerting systems.

Email Setup

By default, Icinga supports email notifications, but we need to configure an email server on the Icinga node as it will be using the Linux mail utilities for sending the mails.

During the initial installation, we installed postfix. Now we need to add google credentials to it so that your server can send mails using Google.

Edit /etc/postfix/main.cf

Change following. ( DO NOT copy paste these in to the end of the file )

# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_tls_security_level=may

smtp_tls_CApath=/etc/ssl/certs
smtp_tls_security_level=may
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

relayhost = [smtp.gmail.com]:587
smtp_use_tls = yes
smtp_sasl_auth_enable = yes
smtp_sasl_security_options =
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt

Create /etc/postfix/sasl_passwd with your Google username and password

[smtp.gmail.com]:587 thilinapathiedulk@gmail.com:MyPassw)<#

where MyPassw)<# would be my password for google. On google you have to make sure, Less Secure Apps are enabled.

now restart postfix

systemctl restart postfix

Test mail transport by mail -s "Hello World" youremail@yourdomain.tld

Create users who needs to be notified

On your Icinga Director web portal, goto Users/Contacts

and then to User Groups. Create a User group as follows by clicking +Add.

Next, Click on the 'Templates' tab under Users/Contacts and create a new template with +Add having the following details.

For the States and Transition Types, you have to select all available, one by one.

Next Go to the Users Tab and add Users of your choice as per the below example. (these users are only for the notification purposes, they cannot log in to Icinga System )

Enable Email Notifications from Director

On Icinga2, we can define custom scripts to execute different notifications. We use this feature to send emails and other notifications. For the simplicity we have created few scripts so it will enable us to put notifications when host or service category makes alerts.

Download email scripts to the scripts directory.

cd /etc/icinga2/scripts

wget https://raw.githubusercontent.com/LEARN-LK/Tutorials/master/Scripts/ICINGA2/icinga2-mail-host.sh

wget https://raw.githubusercontent.com/LEARN-LK/Tutorials/master/Scripts/ICINGA2/icinga2-mail-service.sh

chmod +x icinga2-mail-host.sh

chmod +x icinga2-mail-service.sh

Next, on Icinga Director go to Commands

Click Add to Add new Icinga Command with the following details.

• Command Type: Notification Plugin Command
• Command Name: Host Alarm By Email
• Command: /etc/icinga2/scripts/icinga2-mail-host.sh

and click add. Now go to the Arguments tab of the created command. Then add an Argument like,

Argument name: -4
(Make sure you put the - symbol)
Value type: String
Value: $address$

and Click Add.

Next Click Back button just below the command name on Top of the tab.

Repeat above step and add following arguments as well

At the end of adding all arguments, click Deploy and check whether it deployed without any error.

Then go back to command window and Add another command with following details and arguments.

• Command Type: Notification Plugin Command
• Command Name: Service Alarm By Email
• Command: /etc/icinga2/scripts/icinga2-mail-service.sh
• Arguments;

And Deploy.

Create a Notification Template

Then on the Director, Goto Notifications --> Notification Templates and click Add

Then create a new template to map host based notifications to the script command we created earlier.

Add another template for service based alerts.

Create Notification

Once the templates are created we need to map the to hosts and services individually.

On the Icinga Director, goto Notifications --> Notifications and click Add

Create Notificatoins on Host Alerts to Admins by,

Do the same thing to add a Notification on service alerts.

Above both notifications will apply those settings to any host or service that has notifications enabled. You may customise the Assign where section if you need to customise hosts or notifications that needs this notification to be used.

Finally, we need to Deploy the changes to get them effected.