= Monitoring Netflow Using nfdump/nfsen =
You will install nfdump and nfsen on ubuntu 20.04 in this hands-on.


== install nfdump ==

First, install dependencies
{{{
$ sudo apt-get install build-essential autoconf
$ sudo apt-get install rrdtool mrtg librrds-perl librrdp-perl librrd-dev libmailtools-perl bison flex libbz2-dev libclang-dev
}}}

You might get a box asking '''Make /etc/mrtg.cnf owned by and readable only by root?''' select '''yes'''.

Now proceed to download and build. Note that only the last step (make install) has to be done as root.
{{{
$ cd ~ 
$ git clone https://github.com/phaag/nfdump.git
$ cd nfdump
$ ./autogen.sh
$ ./configure --enable-nfprofile --enable-nftrack --enable-sflow
$ make
$ sudo make install
$ sudo ldconfig
}}}

== Router Configuration ==



=== Cisco Router ===

Let's configure Netflow on a Cisco router
 - Log in to the router and go to configuration mode
{{{
Router>enable
Password: 
#configure terminal
Router(config)#
}}}

 - Create an exporter profile with the IP Address of the server where you have to configure the nfdump and the transport port.
{{{
flow exporter EXPORTER-1
 description Export to nfdump
 destination <IP Address>
 transport udp <port>
 template data timeout 60
}}}

 - Define the flow monitor for IPv4 and call the created exporter. 
{{{
flow monitor FLOW-MONITOR-V4
 exporter EXPORTER-1
 record netflow ipv4 original-input
 cache timeout active 300
}}}

 - Define the flow monitor for IPv6 and call the created exporter.
{{{
flow monitor FLOW-MONITOR-V6
 exporter EXPORTER-1
 record netflow ipv6 original-input
 cache timeout active 300
}}}

 - Add the monitor profile to the Interface that exports the flows
{{{
interface FastEthernet 0/0
 ip flow monitor FLOW-MONITOR-V4 input
 ip flow monitor FLOW-MONITOR-V4 output
  ipv6 flow monitor FLOW-MONITOR-V6 input
  ipv6 flow monitor FLOW-MONITOR-V6 output
}}}

 - Save the configuration

=== HP Router ===
 
 - Login to the router and go to Configure mode
{{{
<Router>system-view
[Router]
}}}

 - Create the sflow agent and sflow source IP. (Interface IP of the interface that exports the flow)
{{{
sflow agent ip <IP Address>
sflow source ip <IP Address>
}}}

 - Define the sflow collector with the nfdump server IP and transport port.
{{{
sflow collector 1 ip <IP Address> port <Port> description "Export to nfdump"
}}}

 - Call the sflow collector from the interface and define other parameters.
{{{
interface GigabitEthernet0/0
 sflow flow collector 1
 sflow sampling-rate 4000
 sflow counter collector 1
 sflow counter interval 120
}}}

 - Save the configuration

== Testing nfcapd/sfcapd and nfdump ==
If your institute's access router uses netflow you should use nfcapd. If it uses sflow you should use sfcapd. 

=== For a netflow router ===
{{{
$ mkdir /tmp/nfcap-test
$ nfcapd -E  -p 9001 -l /tmp/nfcap-test
}}}

... after a while, a series of flows should be dumped on your screen.

Stop the tool with CTRL+C, then look at the contents of /tmp/nfcap-test
{{{
$ ls -l /tmp/nfcap-test
}}}

You should see one or more files called nfcapd.<YEAR><MON><DAY><HR><MIN>

Process the file(s) with nfdump:
{{{
nfdump -r /tmp/nfcap-test/nfcapd.202Ywwxxyyzz | less
nfdump -r /tmp/nfcap-test/nfcapd.202Ywwxxyyzz -s srcip/bytes
}}}
w,x,y,z indicate year, month, day and time.

=== For a sflow router ===
{{{
$ mkdir /tmp/sfcap-test
$ sfcapd -E  -p 9001 -l /tmp/sfcap-test
}}}

... after a while, a series of flows should be dumped on your screen.

Stop the tool with CTRL+C, then look at the contents of /tmp/nfcap-test
{{{
$ ls -l /tmp/sfcap-test
}}}

You should see one or more files called sfcapd.<YEAR><MON><DAY><HR><MIN>

Process the file(s) with nfdump:

{{{
nfdump -r /tmp/sfcap-test/nfcapd.202Ywwxxyyzz | less
nfdump -r /tmp/sfcap-test/nfcapd.202Ywwxxyyzz -s srcip/bytes
}}}
w,x,y,z indicate year, month, day and time.

== Set up folders and nfcapd ==

Following commands are executed as root. You may use `sudo su` to become root.


Create folders per device. 
{{{
mkdir -p /var/nfdump/profiles-data/live/source1/
mkdir -p /var/nfdump/profiles-data/live/source2/
}}}
In the above, a device has been named as a source.

Provide access to the apache2 user:

{{{
chown -R www-data:www-data /var/nfdump/profiles-data
}}}

Assuming device source1 is a netflow device and device source2 is a sflow device, run the following to start the flow collectors.

{{{
/usr/local/bin/nfcapd -w -D -p 9995 -u www-data -g www-data -B 200000 -S 1 -z -I source1 -l /var/nfdump/profiles-data/live/source1/
/usr/local/bin/sfcapd -w -D -p 9996 -u www-data -g www-data -B 200000 -S 1 -z -I source2 -l /var/nfdump/profiles-data/live/source2/
}}}

You can check whether the services are running by observing `netstat -nlp` for open udp port numbers. If they are not working, tail the `/var/log/syslog` for possible errors.

== Installing and setting up NfSen ==

Download and compile nfsen.
{{{
# run following commands as root
# install packages
apt install apache2 git pkg-config php7.4 php7.4-dev libapache2-mod-php7.4 rrdtool librrd-dev
# enable apache modules
a2enmod rewrite deflate headers expires
# install rrd library for php
pecl install rrd 
# create rrd library mod entry for php
echo "extension=rrd.so" > /etc/php/7.4/mods-available/rrd.ini
# enable php mod
phpenmod rrd
# configure virtual host to read .htaccess files
vi /etc/apache2/apache2.conf # set AllowOverride All for /var/www
# restart apache web server
systemctl restart apache2
# install nfsen-ng
cd /var/www/html # or wherever
git clone https://github.com/mbolli/nfsen-ng
chown -R www-data:www-data .
chmod +x nfsen-ng/backend/cli.php
# next step: configuration
}}}

Create the settings file for nfsen-ng

{{{
cp /var/www/html/nfsen-ng/backend/settings/settings.php.dist /var/www/html/nfsen-ng/backend/settings/settings.php
}}}

Edit the settings.php file and include the device names by '''editing''' the following lines.

For the sources:
{{{
        'sources' => array(
            'source1', 'source2',
}}}

For the nfdump:
{{{
    'nfdump' => array(
        'binary' => '/usr/bin/nfdump',
        'profiles-data' => '/var/nfdump/profiles-data',
        'profile' => 'live',
        'max-processes' => 1, // maximum number of concurrently running nfdump processes
    ),
}}}

Next, import existing data:
{{{
/var/www/html/nfsen-ng/backend/cli.php  import
}}}

Now we can start the nfsen-ng daemon:
{{{
/var/www/html/nfsen-ng/backend/cli.php  start
}}}


View flows via the web:

You can find the nfsen-ng page here:

'''http://<your IP address>/nfsen-ng/frontend/'''

In case of server restart, you need to re-run,

{{{
/usr/local/bin/nfcapd -w -D -p 9995 -u www-data -g www-data -B 200000 -S 1 -z -I source1 -l /var/nfdump/profiles-data/live/source1/
/usr/local/bin/sfcapd -w -D -p 9996 -u www-data -g www-data -B 200000 -S 1 -z -I source2 -l /var/nfdump/profiles-data/live/source2/
/var/www/html/nfsen-ng/backend/cli.php  start
}}} 

You may also automate that by keeping a bash script to run on every reboot via cron jobs.


== References: ===

1. https://github.com/mbolli/nfsen-ng
2. https://github.com/phaag/nfdump
3. https://www.systutorials.com/docs/linux/man/1-nfcapd/