= Alerting and Notification on Icinga2 =

Like in LibreNMS, we can setup multiple alerting mechanisms in Icinga2, and since Icinga Stack is a more advanced software, any type of custom notification  system can be associated. In this tutorial, we will only focus on setting up email, slack and telegram alerting systems.

== Email Setup ==

By default, Icinga supports email notifications, but we need to configure an email server on the Icinga node as it will be using the Linux mail utilities for sending the mails.

During the initial installation, we installed `postfix`. Now we need to add google credentials to it so that your server can send mails using Google.

Edit `/etc/postfix/main.cf`

Change following. ( DO NOT copy paste these in to the end of the file )

{{{

# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_tls_security_level=may

smtp_tls_CApath=/etc/ssl/certs
smtp_tls_security_level=may
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

relayhost = [smtp.gmail.com]:587
smtp_use_tls = yes
smtp_sasl_auth_enable = yes
smtp_sasl_security_options =
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt

}}}

Create `/etc/postfix/sasl_passwd` with your Google username and password

{{{
[smtp.gmail.com]:587 thilinapathiedulk@gmail.com:MyPassw)<#
}}}

where `MyPassw)<#` would be my password for google. On google you have to make sure, Less Secure Apps are enabled.

now restart postfix

`systemctl restart postfix`

Test mail transport by `mail -s "Hello World" youremail@yourdomain.tld`

=== Create users who needs to be notified ===

On your Icinga Director web portal, goto `Users/Contacts`

[[Image(Users-Contacts.png, 600px, nolink)]]

and then to  User Groups. Create a `User group` as follows by clicking `+Add`.

[[Image(UserGroup.png, 600px, nolink)]]

Next, Click on the 'Templates' tab under `Users/Contacts` and create a new template with `+Add` having the following details.

[[Image(User-Temp.png, 600px, nolink)]]

For the States and Transition Types, you have to select all available, one by one.

Next Go to the Users Tab and add Users of your choice as per the below example. (these users are only for the notification purposes, they cannot log in to Icinga System )

[[Image(user.png, 600px, nolink)]]


=== Enable Email Notifications from Director ===

On Icinga2, we can define custom scripts to execute different notifications. We use this feature to send emails and other notifications. For the simplicity we have created few scripts so it will enable us to put notifications when host or service category makes alerts.

Download email scripts to the scripts directory.

{{{
#!sh
cd /etc/icinga2/scripts

wget https://raw.githubusercontent.com/LEARN-LK/Tutorials/master/Scripts/ICINGA2/icinga2-mail-host.sh

wget https://raw.githubusercontent.com/LEARN-LK/Tutorials/master/Scripts/ICINGA2/icinga2-mail-service.sh

chmod +x icinga2-mail-host.sh

chmod +x icinga2-mail-service.sh
}}}


Next, on Icinga Director go to Commands 

[[Image(Ici-Di-Commands.png, 400px, nolink)]]

Click `Add` to Add new Icinga Command with the following details.

* Command Type: `Notification Plugin Command`
* Command Name: `Host Alarm By Email`
* Command: `/etc/icinga2/scripts/icinga2-mail-host.sh`

and click add. Now go to the `Arguments` tab of the created command. Then add an Argument like,

Argument name: `-4`[[br]]
(Make sure you put the - symbol)[[br]]
Value type: `String`[[br]]
Value: `$address$`[[br]]


and Click `Add`.

Next Click Back button [[Image(BackBtn.png, 100px, nolink)]] just below the command name on Top of the tab.

Repeat above step and add following arguments as well

[[Image(HostEmail.png, 400px, nolink)]]


At the end of adding all arguments, click `Deploy` and check whether it deployed without any error.

Then go back to command window and Add another command with following details and arguments.


* Command Type: `Notification Plugin Command`
* Command Name: `Service Alarm By Email`
* Command: `/etc/icinga2/scripts/icinga2-mail-service.sh`
* Arguments;

[[Image(Service-Arg.png, 400px, nolink)]]

And Deploy.

=== Create a Notification Template ===

Then on the Director, Goto `Notifications` --> `Notification Templates` and click `Add`

Then create a new template to map host based notifications to the script command we created earlier.

[[Image(Mail-Host-Temp.png, 600px, nolink)]]

Add another template for service based alerts.

[[Image(Mail-Service-Temp.png, 600px, nolink)]]


=== Create Notification ===

Once the templates are created we need to map the to hosts and services individually.

On the Icinga Director, goto `Notifications` --> `Notifications` and click `Add`

Create Notificatoins on Host Alerts to Admins by,

[[Image(Host-Alert-Mail.png, 600px, nolink)]]

Do the same thing to add a Notification on service alerts.

[[Image(Service-Alert-Mail.png, 600px, nolink)]]

Finally, we need to Deploy the changes to get them effected.

[[Image(Deploy.png, 400px, nolink)]]