Understanding Netflow & Nfsen

Most documentation on netflow does not really demonstrate how all the components fit together to provide you with accurate stats/graphs on the netflow sever. This post describes this process using a real world example.

Consider the network diagram below where a single router connects two hosts.




We are going to use SCP on server zzz to copy a file from server abc. This will setup two streams of traffic:

A) zzz (10.48.0.220) -> abc (10.48.254.73) (port 22) - This stream will basically contain all the ACK packets which should be small size packets.

B) abc (10.48.254.73) (port 22) -> zzz (10.48.0.220) - This stream will contain the transferred data and will be large packets.

Assume the netflow server is setup using FreeBSD and Nfsen as described in this post.

To make things harder lets configure both E0/0 and E0/1 to monitor inbound and outbound netflow traffic as shown below:

interface FastEthernet0/0
 description Uplink
 ip flow ingress
 ip flow egress
 !
interface FastEthernet0/1
  description Downlink
 ip flow ingress
 ip flow egress
 !

Start up the scp transfer and on the router you should then see 4 flows on the router as each stream of traffic (A & B) will hit the ingress and the corresponding egress command and generate the flow.

rtr01#show ip cache flow | i 10.48.254.73
Fa0/0         10.48.0.220     Fa0/1*        10.48.254.73    06 A95C 0016    50K
Fa0/0         10.48.0.220     Fa0/1         10.48.254.73    06 A95C 0016    50K
Fa0/1         10.48.254.73    Fa0/0*        10.48.0.220     06 0016 A95C    74K
Fa0/1         10.48.254.73    Fa0/0         10.48.0.220     06 0016 A95C    74K


Now wait for five minutes after the file transfer completes so that nfsen can update correctly. Log into the nfsen server, select the live profile and click on detail:



Then change the Select drop down box from "Single Timeslot" to "Time Window". Use the markers on the graph to set the appropriate Tstart and Tend values.







Then scroll to the bottom of the page and setup the netflow processing as shown below. When you click on the process button you will be presented with the text table below.




This shows larger number of flows than we expected! Lets start at the top:

1) When SSH/SCP run all authentication / protocol negotiation packets are send with a TOS value of zero. This means that the first four flows is really only two streams of traffic which have hit both the inbound and outbound netflow commands. If you are quick enough with the "show ip cache flow" command on the router you can see the same 4 flows there as well.

2) On rtr01 I have configured the command

ip flow-cache timeout active 1

This command means that if you have a long held flow then send an update after every minute as a result the next four flows have a duration of 60.872 seconds which again translates to our two original streams of traffic.

3) The final four flows lasted 22.980 seconds and cover off the last bits of the file transfer.

From this we can deduce the following:

Total trasfer time: 60+22 = 1min 22 sec
Total bytes transferred = 250+93 = 343M
AVG Speed = 32.9 or 32.2 bps = 32.6/8 = 4.075 MB/s

The stats from the SCP transfer confirm these values:


3:44PM user@zzz:~# scp user@abc:~/bigfile a
bigfile                            100%  329MB   3.9MB/s   01:25


The question remains: Why would you want to monitor ingress/egress for all interfaces?

In router configurations you may turn on link compression on some interfaces. This means you could have 100M enter one interface and 90M exit the other side.


If we were monitoring only ingress traffic then we will not be able to see this discrepancy.

Commercial netflow monitoring software have enough smarts to "deduplicate" these flow and only present the real information. In the case of nfsen, I would recommend that you monitor only the inbound traffic on all interfaces.