Monitoring Netflow Using nfdump/nfsen
You will install nfdump and nfsen on ubuntu 20.04 in this hands-on.
install nfdump
First, install dependencies
$ sudo apt-get install build-essential autoconf $ sudo apt-get install rrdtool mrtg librrds-perl librrdp-perl librrd-dev libmailtools-perl bison flex libbz2-dev libclang-dev
You might get a box asking Make /etc/mrtg.cnf owned by and readable only by root? select yes.
Now proceed to download and build. Note that only the last step (make install) has to be done as root.
$ cd ~ $ git clone https://github.com/phaag/nfdump.git $ cd nfdump $ ./autogen.sh $ ./configure --enable-nfprofile --enable-nftrack --enable-sflow $ make $ sudo make install $ sudo ldconfig
Router Configuration
Cisco Router
Let's configure Netflow on a Cisco router
- Log in to the router and go to configuration mode
Router>enable Password: #configure terminal Router(config)#
- Create an exporter profile with the IP Address of the server where you have to configure the nfdump and the transport port.
flow exporter EXPORTER-1 description Export to nfdump destination <IP Address> transport udp <port> template data timeout 60
- Define the flow monitor for IPv4 and call the created exporter.
flow monitor FLOW-MONITOR-V4 exporter EXPORTER-1 record netflow ipv4 original-input cache timeout active 300
- Define the flow monitor for IPv6 and call the created exporter.
flow monitor FLOW-MONITOR-V6 exporter EXPORTER-1 record netflow ipv6 original-input cache timeout active 300
- Add the monitor profile to the Interface that exports the flows
interface FastEthernet 0/0 ip flow monitor FLOW-MONITOR-V4 input ip flow monitor FLOW-MONITOR-V4 output ipv6 flow monitor FLOW-MONITOR-V6 input ipv6 flow monitor FLOW-MONITOR-V6 output
- Save the configuration
HP Router
- Login to the router and go to Configure mode
<Router>system-view [Router]
- Create the sflow agent and sflow source IP. (Interface IP of the interface that exports the flow)
sflow agent ip <IP Address> sflow source ip <IP Address>
- Define the sflow collector with the nfdump server IP and transport port.
sflow collector 1 ip <IP Address> port <Port> description "Export to nfdump"
- Call the sflow collector from the interface and define other parameters.
interface GigabitEthernet0/0 sflow flow collector 1 sflow sampling-rate 4000 sflow counter collector 1 sflow counter interval 120
- Save the configuration
Testing nfcapd/sfcapd and nfdump
If your institute's access router uses netflow you should use nfcapd. If it uses sflow you should use sfcapd.
For a netflow router
$ mkdir /tmp/nfcap-test $ nfcapd -E -p 9001 -l /tmp/nfcap-test
... after a while, a series of flows should be dumped on your screen.
Stop the tool with CTRL+C, then look at the contents of /tmp/nfcap-test
$ ls -l /tmp/nfcap-test
You should see one or more files called nfcapd.<YEAR><MON><DAY><HR><MIN>
Process the file(s) with nfdump:
nfdump -r /tmp/nfcap-test/nfcapd.202Ywwxxyyzz | less nfdump -r /tmp/nfcap-test/nfcapd.202Ywwxxyyzz -s srcip/bytes
w,x,y,z indicate year, month, day and time.
For a sflow router
$ mkdir /tmp/sfcap-test $ sfcapd -E -p 9001 -l /tmp/sfcap-test
... after a while, a series of flows should be dumped on your screen.
Stop the tool with CTRL+C, then look at the contents of /tmp/nfcap-test
$ ls -l /tmp/sfcap-test
You should see one or more files called sfcapd.<YEAR><MON><DAY><HR><MIN>
Process the file(s) with nfdump:
nfdump -r /tmp/sfcap-test/nfcapd.202Ywwxxyyzz | less nfdump -r /tmp/sfcap-test/nfcapd.202Ywwxxyyzz -s srcip/bytes
w,x,y,z indicate year, month, day and time.
Set up folders and nfcapd
Following commands are executed as root. You may use sudo su to become root.
Create folders per device.
mkdir -p /var/nfdump/profiles-data/live/source1/ mkdir -p /var/nfdump/profiles-data/live/source2/
In the above, a device has been named as a source.
Provide access to the apache2 user:
chown -R www-data:www-data /var/nfdump/profiles-data
Assuming device source1 is a netflow device and device source2 is a sflow device, run the following to start the flow collectors.
/usr/local/bin/nfcapd -w -D -p 9995 -u www-data -g www-data -B 200000 -S 1 -z -I source1 -l /var/nfdump/profiles-data/live/source1/ /usr/local/bin/sfcapd -w -D -p 9996 -u www-data -g www-data -B 200000 -S 1 -z -I source2 -l /var/nfdump/profiles-data/live/source2/
You can check whether the services are running by observing netstat -nlp for open udp port numbers. If they are not working, tail the /var/log/syslog for possible errors.
Installing and setting up NfSen
Download and compile nfsen.
# run following commands as root # install packages apt install apache2 git pkg-config php7.4 php7.4-dev libapache2-mod-php7.4 rrdtool librrd-dev # enable apache modules a2enmod rewrite deflate headers expires # install rrd library for php pecl install rrd # create rrd library mod entry for php echo "extension=rrd.so" > /etc/php/7.4/mods-available/rrd.ini # enable php mod phpenmod rrd # configure virtual host to read .htaccess files vi /etc/apache2/apache2.conf # set AllowOverride All for /var/www # restart apache web server systemctl restart apache2 # install nfsen-ng cd /var/www/html # or wherever git clone https://github.com/mbolli/nfsen-ng chown -R www-data:www-data . chmod +x nfsen-ng/backend/cli.php # next step: configuration
Create the settings file for nfsen-ng
cp /var/www/html/nfsen-ng/backend/settings/settings.php.dist /var/www/html/nfsen-ng/backend/settings/settings.php
Edit the settings.php file and include the device names by editing the following lines.
For the sources:
'sources' => array(
'source1', 'source2',
For the nfdump:
'nfdump' => array(
'binary' => '/usr/bin/nfdump',
'profiles-data' => '/var/nfdump/profiles-data',
'profile' => 'live',
'max-processes' => 1, // maximum number of concurrently running nfdump processes
),
Next, import existing data:
/var/www/html/nfsen-ng/backend/cli.php import
Now we can start the nfsen-ng daemon:
/var/www/html/nfsen-ng/backend/cli.php start
View flows via the web:
You can find the nfsen-ng page here:
http://<your IP address>/nfsen-ng/frontend/
In case of server restart, you need to re-run,
/usr/local/bin/nfcapd -w -D -p 9995 -u www-data -g www-data -B 200000 -S 1 -z -I source1 -l /var/nfdump/profiles-data/live/source1/ /usr/local/bin/sfcapd -w -D -p 9996 -u www-data -g www-data -B 200000 -S 1 -z -I source2 -l /var/nfdump/profiles-data/live/source2/ /var/www/html/nfsen-ng/backend/cli.php start
You may also automate that by keeping a bash script to run on every reboot via cron jobs.
