wiki:NspwUprouse/Agenda/netflow

Monitoring Netflow Using nfdump/nfsen

You will install nfdump and nfsen on ubuntu 20.04 in this hands-on.

install nfdump

First, install dependencies

$ sudo apt-get install build-essential autoconf
$ sudo apt-get install rrdtool mrtg librrds-perl librrdp-perl librrd-dev libmailtools-perl bison flex libbz2-dev libclang-dev

You might get a box asking Make /etc/mrtg.cnf owned by and readable only by root? select yes.

Now proceed to download and build. Note that only the last step (make install) has to be done as root.

$ cd ~ 
$ git clone https://github.com/phaag/nfdump.git
$ cd nfdump
$ ./autogen.sh
$ ./configure --enable-nfprofile --enable-nftrack --enable-sflow
$ make
$ sudo make install
$ sudo ldconfig

Router Configuration

Cisco Router

Let's configure Netflow on a Cisco router

  • Log in to the router and go to configuration mode
    Router>enable
    Password: 
    #configure terminal
    Router(config)#
    
  • Create an exporter profile with the IP Address of the server where you have to configure the nfdump and the transport port.
    flow exporter EXPORTER-1
     description Export to nfdump
     destination <IP Address>
     transport udp <port>
     template data timeout 60
    
  • Define the flow monitor for IPv4 and call the created exporter.
    flow monitor FLOW-MONITOR-V4
     exporter EXPORTER-1
     record netflow ipv4 original-input
     cache timeout active 300
    
  • Define the flow monitor for IPv6 and call the created exporter.
    flow monitor FLOW-MONITOR-V6
     exporter EXPORTER-1
     record netflow ipv6 original-input
     cache timeout active 300
    
  • Add the monitor profile to the Interface that exports the flows
    interface FastEthernet 0/0
     ip flow monitor FLOW-MONITOR-V4 input
     ip flow monitor FLOW-MONITOR-V4 output
      ipv6 flow monitor FLOW-MONITOR-V6 input
      ipv6 flow monitor FLOW-MONITOR-V6 output
    
  • Save the configuration

HP Router

  • Login to the router and go to Configure mode
    <Router>system-view
    [Router]
    
  • Create the sflow agent and sflow source IP. (Interface IP of the interface that exports the flow)
    sflow agent ip <IP Address>
    sflow source ip <IP Address>
    
  • Define the sflow collector with the nfdump server IP and transport port.
    sflow collector 1 ip <IP Address> port <Port> description "Export to nfdump"
    
  • Call the sflow collector from the interface and define other parameters.
    interface GigabitEthernet0/0
     sflow flow collector 1
     sflow sampling-rate 4000
     sflow counter collector 1
     sflow counter interval 120
    
  • Save the configuration

Testing nfcapd/sfcapd and nfdump

If your institute's access router uses netflow you should use nfcapd. If it uses sflow you should use sfcapd.

For a netflow router

$ mkdir /tmp/nfcap-test
$ nfcapd -E  -p 9001 -l /tmp/nfcap-test

... after a while, a series of flows should be dumped on your screen.

Stop the tool with CTRL+C, then look at the contents of /tmp/nfcap-test

$ ls -l /tmp/nfcap-test

You should see one or more files called nfcapd.<YEAR><MON><DAY><HR><MIN>

Process the file(s) with nfdump:

nfdump -r /tmp/nfcap-test/nfcapd.202Ywwxxyyzz | less
nfdump -r /tmp/nfcap-test/nfcapd.202Ywwxxyyzz -s srcip/bytes

w,x,y,z indicate year, month, day and time.

For a sflow router

$ mkdir /tmp/sfcap-test
$ sfcapd -E  -p 9001 -l /tmp/sfcap-test

... after a while, a series of flows should be dumped on your screen.

Stop the tool with CTRL+C, then look at the contents of /tmp/nfcap-test

$ ls -l /tmp/sfcap-test

You should see one or more files called sfcapd.<YEAR><MON><DAY><HR><MIN>

Process the file(s) with nfdump:

nfdump -r /tmp/sfcap-test/nfcapd.202Ywwxxyyzz | less
nfdump -r /tmp/sfcap-test/nfcapd.202Ywwxxyyzz -s srcip/bytes

w,x,y,z indicate year, month, day and time.

Set up folders and nfcapd

Following commands are executed as root. You may use sudo su to become root.

Create folders per device.

mkdir -p /var/nfdump/profiles-data/live/source1/
mkdir -p /var/nfdump/profiles-data/live/source2/

In the above, a device has been named as a source.

Provide access to the apache2 user:

chown -R www-data:www-data /var/nfdump/profiles-data

Assuming device source1 is a netflow device and device source2 is a sflow device, run the following to start the flow collectors.

/usr/local/bin/nfcapd -w -D -p 9995 -u www-data -g www-data -B 200000 -S 1 -z -I source1 -l /var/nfdump/profiles-data/live/source1/
/usr/local/bin/sfcapd -w -D -p 9996 -u www-data -g www-data -B 200000 -S 1 -z -I source2 -l /var/nfdump/profiles-data/live/source2/

You can check whether the services are running by observing netstat -nlp for open udp port numbers. If they are not working, tail the /var/log/syslog for possible errors.

Installing and setting up NfSen?

Download and compile nfsen.

# run following commands as root
# install packages
apt install apache2 git pkg-config php7.4 php7.4-dev libapache2-mod-php7.4 rrdtool librrd-dev
# enable apache modules
a2enmod rewrite deflate headers expires
# install rrd library for php
pecl install rrd 
# create rrd library mod entry for php
echo "extension=rrd.so" > /etc/php/7.4/mods-available/rrd.ini
# enable php mod
phpenmod rrd
# configure virtual host to read .htaccess files
vi /etc/apache2/apache2.conf # set AllowOverride All for /var/www
# restart apache web server
systemctl restart apache2
# install nfsen-ng
cd /var/www/html # or wherever
git clone https://github.com/mbolli/nfsen-ng
chown -R www-data:www-data .
chmod +x nfsen-ng/backend/cli.php
# next step: configuration

Create the settings file for nfsen-ng

cp /var/www/html/nfsen-ng/backend/settings/settings.php.dist /var/www/html/nfsen-ng/backend/settings/settings.php

Edit the settings.php file and include the device names by editing the following lines.

For the sources:

        'sources' => array(
            'source1', 'source2',

For the nfdump:

    'nfdump' => array(
        'binary' => '/usr/bin/nfdump',
        'profiles-data' => '/var/nfdump/profiles-data',
        'profile' => 'live',
        'max-processes' => 1, // maximum number of concurrently running nfdump processes
    ),

Next, import existing data:

/var/www/html/nfsen-ng/backend/cli.php  import

Now we can start the nfsen-ng daemon:

/var/www/html/nfsen-ng/backend/cli.php  start

View flows via the web:

You can find the nfsen-ng page here:

http://<your IP address>/nfsen-ng/frontend/

In case of server restart, you need to re-run,

/usr/local/bin/nfcapd -w -D -p 9995 -u www-data -g www-data -B 200000 -S 1 -z -I source1 -l /var/nfdump/profiles-data/live/source1/
/usr/local/bin/sfcapd -w -D -p 9996 -u www-data -g www-data -B 200000 -S 1 -z -I source2 -l /var/nfdump/profiles-data/live/source2/
/var/www/html/nfsen-ng/backend/cli.php  start

You may also automate that by keeping a bash script to run on every reboot via cron jobs.

References: =

  1. https://github.com/mbolli/nfsen-ng
  2. https://github.com/phaag/nfdump
  3. https://www.systutorials.com/docs/linux/man/1-nfcapd/
Last modified 13 months ago Last modified on Sep 13, 2021, 2:41:47 AM