≡ Menu

default output

Apache Web Server: Log Analysis and Server Status Monitoring Tool

wtop is really cool application for web server log analysis and to see server stats at a glance. It also has powerful log grepping capability. It is just like 'top' for your webserver.

It can find out number of searches or signups per seconds. It can also create histogram of response time. There is also another tool called logrep a powerful command-line program for ad-hoc analysis and filtering for log files. You can dig up lots of information using wtop tools.

You need Python version 2.5 to run wtop.

Download wtop

Type the following command:
$ cd /tmp
$ wget http://wtop.googlecode.com/files/wtop-0.5.6.tar.gz
$ tar -zxvf wtop-0.5.6.tar.gz
$ cd wtop-0.5.6
# python setup.py install

Configuring wtop

Once installed you can start using the tool immediately. You need to edit /etc/wtop.cfg file to setup parameters, Apache log files and other directives
# vi /etc/wtop.cfg
Sample configuration file:

# This must match your webserver log format. You MUST have at least %h, %r and %D
LOG_FORMAT=%h %l %u %t "%r" %>s %B "%{Referer}i" "%{User-Agent}i" %D
# max time before a request is logged in the "slow" column
# minimum requests/second before a URL class appears in top mode
# you can extend these to make any classes you wish
# the generic pattern is applied if a line does not match any
# of the named classes. By default it uses the top-level directory.
# incomplete list of known web robots
robots = r'(?:nutch|MSRBOT|translate.google.com|Feedster|Nutch|Gaisbot|Snapbot|VisBot|libwww|CazoodleBot|polybot|VadixBot|Sogou|SBider|BecomeBot|Yandex|Pagebull|chudo|Pockey|nicebot|entireweb|FeedwhipBOT|ConveraCrawler|NG/2.0|WebImages|Factbot|information-online|gsa-crawler|Jyxobot|SentinelCrawler|BlogPulseLive|YahooFeedSeeker|GurujiBot|wwwster|Y\!J-SRD|Findexa|SurveyBot|yetibot|discoveryengine|fastsearch|noxtrum|Googlebot|Snapbot|OGSearchSpider|heritrix|nutch-agent|Slurp|msnbot|cuill|Mediapartners|YahooSeeker|GrabPERF|keywen|ia_archiver|crawler.archive.org|Baiduspider|larbin|shopwiki)'

Now simply type wtop at a shell prompt:
$ wtop$
See all human traffic, enter:
$ logrep -m top -h access.log
See response times for all MSNBot homepage hits:
$ logrep -m grep -g MSNBot -i home -o status,msec,url access.log
Display the current log for traffic to pages about wordpress or themes sent from google.com
$ logrep -m tail --f 'url~wordpress|themes,ref~google.com' access.log

Further readings:

Linux and UNIX interactive, process and users monitoring tool

whowatch is a interactive, ncurses-based, process and users monitoring tool, which updates information in real time. This is a perfect tool for local and remote servers. With this tool you can easily answer following question:
"How do I know who are logged on in using telnet , ssh, ftp etc and what resources are they are using?"

Output of whowatch command

It displays information about the users currently logged on to the machine, in real-time. Besides standard information (login name, tty, host, user's process), the type of the connection (ie. telnet or ssh) is shown. Display of users command line can be switch
to tty idle time. Certain user can be selected and his processes tree may be viewed as well as tree of all system processes. Tree may be displayed with additional column that shows owner of each process. In the process tree mode SIGINT and SIGKILL signals can be sent to the selected process. Killing processes is just as simple and fun as deleting lines on the screen.

How do I install whowatch tool?

If you are using Debian Linux, type the following command:
# apt-get install whowatch

If you are using FreeBSD, type the any one of the following command:
# pkg_add -r -v whowatch

You can also use ports collection under FreeBSD:
# cd /ports/sysutils/whowatch
# make; make install; make clean

ALTERNATIVELY, download from official website.

How do I use whowatch?

Simply type whowatch at command prompt:
$ whowatch

Default output:
Who watch output

Detailed information about process / user


Menu (press F9 key to activate menu option)