Peter Tran – Sydney, Australia

I was recently asked to help assist in determining why a client’s network was downloading more than 8GB per day which appeared to be happening for almost a week. The client, being on an Internode ISP connection would’ve been charged 0.05c per megabyte on top of their 60GB of allocated bandwidth quota per month. The issue was that within the SOHO business, that either:

1. Someone was downloading large files within the organisation
2. Updates were occurring on their servers (over 20 virtualised servers) on a daily basis
3. Large backups were being transferred across the Internet, or
4. Something else is chewing up the bandwidth.

Analysing the Internode usage graphs, they noticed that there was a constant 350-400MB download occurring every hour. The client even went to extreme measures and turned off all workstations in the office during the weekend and left the servers on, yet still something was downloading.

Being unsuccessful in locating where the downloads could have went, they’ve asked me to take a look at their servers. After three hours of analysis, it all came down to the Kayako Help Desk software, attempting to fetch large mails from the mail server that was larger than what the PHP script could handle.

A Cronjob installed on the Ubuntu server within the organisation runs a script every 5 minutes to fetch emails via a designated POP account. The script attempts to download a 30MB file, when deep within the php.ini settings on the server, it’s maximum ‘memory_limit’ setting was 16MB large. Therefore the download of this file failed, every 5 minutes, yet still manages to download 30MBs each time. Do some calculations, that means 360MB is downloaded per hour, equating to 8.4GB being downloaded per day. I got them to delete the large email, and I increased the PHP memory_limit size to 32MB to ensure that this does not happen again.

How did I pick this up?

Looking for a suitable bandwidth network meter for Linux, I came across vnStat that did the job just fine. To install this onto an Ubuntu Server, run:

sudo apt-get install vnstat

Make a note of which interfaces you want to run the metering service on:

ifconfig

In my case, I had to monitor eth0 and eth1. To run the cron job (scheduled task) that creates a task that will run every few minutes and collect traffic data for vnStat, enter:

vnstat -u -i eth0

vnstat -u -i eth1

To check the status, type vnstat. Below is a typical output after running the command:

                     rx      /     tx      /    total    /  estimated
 eth1:
     yesterday     27.43 MB  /   47.96 MB  /   75.39 MB
         today     11.89 MB  /   18.72 MB  /   30.61 MB  /      63 MB

 eth0:
     yesterday    177.98 MB  /  192.08 MB  /  370.05 MB
         today     94.63 MB  /   82.01 MB  /  176.64 MB  /     386 MB

There are more display options such as viewing hourly statistics, which you can view by entering vnstat -h. For more options, see man vnstat.

If you prefer to view the usage stats from your web browser, you can use Squeek’s vnStat PHP Frontend. To install, download upload the contents to the latest release, upload to the root folder of your apache server. For simplicity sake, I upload it to a folder such as e.g. http://www.yourwebsiteurl.com/vnstat for easy access.

To configure the script:

sudo nano /path/to/vnstat/config.php

Modify the following lines to reflect your requirements. I modified the below lines to meet my needs:

// list of network interfaces monitored by vnStat
$iface_list = array('eth0', 'eth1');

//
// optional names for interfaces
// if there's no name set for an interface then the interface identifier
// will be displayed instead
//
$iface_title['eth0'] = 'Internode Connection';
$iface_title['eth1'] = 'Exetel Connection';

//
// There are two possible sources for vnstat data. If the $vnstat_bin
// variable is set then vnstat is called directly from the PHP script
// to get the interface data.
//
// The other option is to periodically dump the vnstat interface data to
// a file (e.g. by a cronjob). In that case the $vnstat_bin variable
// must be cleared and set $data_dir to the location where the dumps
// are stored. Dumps must be named 'vnstat_dump_$iface'.
//
// You can generate vnstat dumps with the command:
// vnstat --dumpdb -i $iface > /path/to/data_dir/vnstat_dump_$iface
//
$vnstat_bin = '/usr/bin/vnstat';

Save the settings, and you should have a pretty nifty network meter tool to gauge the usages over a period of time.