Apache / Nginx: Visualize Web Server Access Log In Real Time

by on May 1, 2013 · 12 comments· LAST UPDATED May 5, 2013

in Debian Linux, Open Source, Web Developer

Logstalgia (also known as ApachePong) is a very powerful and handy app. It is a website access log visualization tool. It is an extremely useful tool to give you look at your web server traffic. I often use this kind of software to justify and convince my clients and/or boss that we need more servers to handle traffic. It streams Apache / Lighttpd / Nginx web-server access logs as a pong-like battle between the web server and a never ending torrent of requests.



From the official project site:

Logstalgia is a website traffic visualization that replays web-server access logs as a pong-like battle between the web server and an never ending torrent of requests. Requests appear as colored balls (the same color as the host) which travel across the screen to arrive at the requested location. Successful requests are hit by the paddle while unsuccessful ones (eg 404 - File Not Found) are missed and pass through. The paths of requests are summarized within the available space by identifying common path prefixes. Related paths are grouped together under headings. For instance, by default paths ending in png, gif or jpg are grouped under the heading Images. Paths that don’t match any of the specified groups are lumped together under a Miscellaneous section.

Logstalgia requirements

  1. Workstation/Laptop/Server with a video card supporting OpenGL.
  2. A a fairly busy web-server to see interesting results.
  3. Linux operating systems. I tested this app on both Fedora and Debian/Ubuntu Linux.

Binary package based installation

Open a terminal and type the following command:
$ sudo apt-get install logstalgia
Sample outputs:

Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  logstalgia
0 upgraded, 1 newly installed, 0 to remove and 4 not upgraded.
Need to get 161 kB of archives.
After this operation, 1,102 kB of additional disk space will be used.
Get:1 http://mirrors.kernel.org/debian/ stable/main logstalgia amd64 1.0.0-1+b1 [161 kB]
Fetched 161 kB in 2s (73.9 kB/s)
Selecting previously deselected package logstalgia.
(Reading database ... 338532 files and directories currently installed.)
Unpacking logstalgia (from .../logstalgia_1.0.0-1+b1_amd64.deb) ...
Processing triggers for man-db ...
Setting up logstalgia (1.0.0-1+b1) ...

Source code installation

You need to install the following software to build logstalgia. Open a terminal and type the following command:
$ sudo apt-get install libsdl1.2-dev libsdl-image1.2-dev libpcre3-dev libftgl-dev libpng12-dev libjpeg62-dev

Download and build the software

Type the following wget command to download the software:
$ cd /tmp
$ wget https://logstalgia.googlecode.com/files/logstalgia-1.0.3.tar.gz

Sample outputs:

--2013-04-30 15:17:41--  https://logstalgia.googlecode.com/files/logstalgia-1.0.3.tar.gz
Resolving logstalgia.googlecode.com... 74.125.135.82, 2404:6800:4001:c01::52
Connecting to logstalgia.googlecode.com|74.125.135.82|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 841822 (822K) [application/x-gzip]
Saving to: `logstalgia-1.0.3.tar.gz'
 
100%[======================================>] 8,41,822     456K/s   in 1.8s
 
2013-04-30 15:17:44 (456 KB/s) - `logstalgia-1.0.3.tar.gz' saved [841822/841822]

Untar the tar ball with the tar command, enter:
$ tar xvf logstalgia-1.0.3.tar.gz
First, make sure you have gcc compiler and related tools are installed. Type the following command to build software:
$ cd logstalgia
$ ./configure

To compile logstalgia, type the following command:
$ make
Finally, install logstalgia, enter:
$ sudo make install
Sample outputs:

make[1]: Entering directory `/tmp/logstalgia-1.0.3'
test -z "/usr/local/bin" || /bin/mkdir -p "/usr/local/bin"
  /usr/bin/install -c logstalgia '/usr/local/bin'
test -z "/usr/local/share/logstalgia/fonts" || /bin/mkdir -p "/usr/local/share/logstalgia/fonts"
 /usr/bin/install -c -m 644 data/fonts/README data/fonts/FreeMonoBold.ttf data/fonts/FreeSerif.ttf '/usr/local/share/logstalgia/fonts'
test -z "/usr/local/share/logstalgia" || /bin/mkdir -p "/usr/local/share/logstalgia"
 /usr/bin/install -c -m 644 data/ball.tga data/example.log data/glow.tga '/usr/local/share/logstalgia'
make  install-data-hook
make[2]: Entering directory `/tmp/logstalgia-1.0.3'
mkdir -p -m 755 //usr/local/share/man/man1
/bin/sed 's|SDLAPP_RESOURCE_DIR|/usr/local/share/logstalgia|g' data/logstalgia.1 | gzip -f9 > /usr/local/share/man/man1/logstalgia.1.gz
make[2]: Leaving directory `/tmp/logstalgia-1.0.3'
make[1]: Leaving directory `/tmp/logstalgia-1.0.3'

How do I use logstalgia?

The syntax is as follows:

 
logstalgia access_log
logstalgia /var/www/httpd/access_log
logstalgia [options] /var/www/httpd/access_log
 

Examples

To watch /var/www/apache2/access.log file using the default settings, enter:
$ logstalgia /var/www/apache2/access.log
You can force logstalgia to read logs from STDIN:
$ tail -f /var/log/httpd/access_log | logstalgia -
OR
$ tail -f /var/log/httpd/access_log | logstalgia --sync

How do I watch the live access.log from the remote server?

Use the following syntax over the ssh session for piracy and security reasons:

 
ssh vivek@server1.cyberciti.biz tail -f /var/log/apache2/access.log | logstalgia --sync
ssh vivek@server1.cyberciti.biz tail -f /var/log/nginx/www.cyberciti.bizaccess.log | logstalgia --sync
 

How do I save logstalgia output as a video file?

You can create a video of Logstalgia using the --output-ppm-stream option. The syntax is to create ppm file:
$ logstalgia -1280x720 --output-ppm-stream output.ppm /var/log/httpd/access.log
OR use the ssh command for remote web-server:
$ ssh user@server1.cyberciti.biz tail -f /var/log/nginx/www.cyberciti.biz_access.log | logstalgia -1280x720 --output-ppm-stream --sync output.ppm
Finally, use the ffmpeg command as follows to create .mp4 file from .ppm file:
$ ffmpeg -y -r 60 -f image2pipe -vcodec ppm -i output.ppm -vcodec libx264 -preset ultrafast -pix_fmt yuv420p -crf 1 -threads 0 -bf 0 nginx.server.log.mp4
To play the file nginx.server.log.mp4 use mplayer command, type:
$ mplayer nginx.server.log.mp4

Keyboard shortcuts

Interactive keyboard commands are follows:

  1. c : Displays Logstalgia logo.
  2. n : Jump forward in time to next log entry.
  3. + or - : Adjust simulation speed.
  4. < or > : Adjust time scale.
  5. Esc : Quit the app.

Type the following command to see all other supported options:
$ logstalgia --help

Check out related media

This tutorial is also available in a quick video format:



Video 01: Linux / Unix Web Server Log File Visualization in Realtime With Logstalgia

Other realtime logfile visualization software for *nix

  1. glTail.rb - a real-time logfile visualization written in Ruby.
  2. goaccess - a real-time open source web log analyzer and interactive viewer.
References
TwitterFacebookGoogle+PDF versionFound an error/typo on this page? Help us!

  • william dumangeng

    RHEL/Centos ? no ?

  • Guest

    I have to add to WIlliam’s question. The configure is not smart enough to see that the SDL is installed in the lib64 infra, so you need to go through all sorts of hoops just to make the thing compile. It’d be nice to get more system support. Something as ubiquitous as Apache (which literally runs on EVERY platform) should have a RedHat aware configure that knows where to look for libraries. :(

  • Scott

    I’ve tried three different Linuxes on Virtual box all with no success Best, I’ve gotten in Ubuntu and I get

    terminate called after throwing an instance of ‘SDLInitException’
    what(): Couldn’t find matching GLX visual
    Aborted (core dumped)

  • nixcraft

    You need OpenGL compatible graphics card. It will not work in Virual box or any other VM. Or your VM must support OpenGL.

  • nixcraft

    @Guest and @william I will see if I can compile this one on CentOS/RHEL.

  • Matt Adams

    I think I will try GoAccess, looks pretty neat though. Installing OpenGL in a server is not an option for me. Thanks for sharing!

  • vonskippy

    ” It is an extremely useful tool to give you look at your web server traffic”

    I don’t see where this is anything but a shiny object to wow people too stupid to look at simple stats.

    How does a bunch of flying pixels moving back and forth tell me MORE CLEARLY how much traffic my web server has then a simple report reading 382 hits/sec does?

    Graphics are great for comparing multiple complex objects – besides traffic, what is this snowflake swirl comparing?

    It looks pretty, but after several minutes thinking about it, I still can’t come up with a business case to use it.

    GoAccess looks far more useful (but I’m not sure that project is still alive).

  • theMusician5044

    If you are on a mac and use homebrew you can install it with

    $ homebrew install logstalgia

  • shams tarek

    sooooo funny .. i love it

  • Jeff

    That looked pretty cool !! Goaccess looked Blah !!! P.S. a few of us also liked the music.

  • justme

    Nice, but development has stopped since feb 2011…… that’s not good

  • Paul Anderson

    GoAccess just released v0.7, woot!

Previous post:

Next post: