Download of the day: phpMyVisites free and open source websites statistics and analytics software

Posted on in Categories Apache, Download of the day, lighttpd, Linux, UNIX, Windows server last updated October 3, 2007

phpMyVisites is a free and powerful open source (GNU/GPL) software for websites statistics and audience measurements software. I’m currently using this software and it totally rocks. This software gives out lots of information on websites visitors, visited pages, software/hardware utilization. The installation is entirely automated and very simple. I’m currently using the same software here. This software is much better than old AWstats package. Web analytics is the study of the behaviour of website visitors. In a commercial context, web analytics especially refers to the use of data collected from a web site to determine which aspects of the website work towards the business objectives; for example, which landing pages encourage people to make a purchase.

From the project home page:

phpMyVisites is web statistics software. It is also often called web analytics. phpMyVisites is open source and free. You can download it, install it on your webserver, and get your first statistics after 2 minutes! Then all these numbers may be very useful to improve your website results. If you understand how your visitors behave, if you try to analyse your audience and extract information from the web analytics reports, you can definitely boost your website!

Software features

  • A clean and user-friendly interface to present data and to aid in data analysis.
  • Clear and concise graphics presenting important information in an easy-to-understand format.
  • Free: phpMyVisites is completely free.
  • Precise visitor statistics over a period of time (day/week/month/year).
  • Visitor Frequency: new visitors, regular (known) visitors, and how often visitors view the web site.
  • Management of web site statistics and all file types (PDF, Image, etc.).
  • Web site page classification available (by groups, by subgroups, etc.).
  • Visitor Analysis: Statistics for pages where visitors leave the web site and for pages where the visitors enter the web site.
  • Geographical Statistics: Classification by continent/country (interactive world map).
  • Technical Configuration Statistics: Web browsers, resolution, managed plug-in, etc.).
  • Complete and clear statistics about web site discovery: How do visitors come to the web site?
  • Live Clearly Defined Web Site Discovery Tools: Search Engines, Web Sites, Partner Sites, Newsletters and Direct Access
  • Able to detect more than 300 internationally-used search engines and keyword associations.
  • Define web sites as partners and add an unlimited number of newsletters.
  • One software installation and track all your website
  • Receive web site statistics everyday by e-mail, by RSS feed, etc.
  • And much more..

phpMyVisites free and open source websites statistics and analytics software

phpMyVisites free and open source websites statistics and analytics software
You can see sample reports and screenshots here

Download phpMyVisites

You need a webserver such as Apache, Lighttpd, IIS, etc.) that supports the following :

  1. php > 4.3
  2. Mysql database
  3. GD Library
  4. TTF support (Freetype) etc

=> Visit official site to download phpMyVisites software.

Troubleshooting: Slow Apache / IIS / Lighttpd Webserver Website Problems

Posted on in Categories Apache, Howto, lighttpd, Linux, Troubleshooting, UNIX last updated October 2, 2007

A small how to written to assist sys admins. It offers tools and high level advice to solve slow websites problems:

Oh boy it’s been an intense 4 weeks. I was pulled in to assist with troubleshooting several major failures at work with loosely related systems. Including my own. All systems are public facing, hence the intensity.

This article attempts to capture a high level view of one of the problems and the methods and tools used (not how to use the tools, that’s for another time) to try and solve it. I’m not putting all the details in. Some details are emphatically not appropriate, other details won’t increase understanding of the methodology and process taken.

In the interest of helping other sysadmins, I’ve linked to the various tools used. Some are well known, others more obscure.

=> Advanced Sysadmin Troubleshooting: Slow Websites (via RootPrompt)

How to upgrade lighttpd tar ball (source code) installation

Posted on in Categories CentOS, Howto, lighttpd, Linux, RedHat/Fedora Linux, Security, Suse Linux, Sys admin, Tips, Ubuntu Linux last updated September 1, 2007

Upgrading lighttpd is a piece of cake. There are two methods:

a) Use yum or apt-get or FreeBSD ports / command to update binary lighttpd package

b) Just download latest lighttpd tar ball from official web site and install the same.

Let us see how to upgrade lighttpd using source code (tar ball).

# 1 : Download lighttpd

Use wget or lftp command line http / ftp accelerator tools:
$ cd /opt
$ wget http://www.lighttpd.net/download/lighttpd-1.4.17.tar.gz

# 2 : Verify lighttpd

Use sha1sum or md5sum hash to verify lighttpd tar ball integrity:
$ md5sum lighttpd-1.4.17.tar.gz

# 3: Configure lighttpd

Now configure and compile lighttpd web server:
$ ./configure
$ make

# 4: Stop lighttpd

First stop currently running lighttpd web server:
# /etc/init.d/lighttpd stop
Make sure you are in installation directory, use the following command to uninstall old version:
# make uninstall

# 5: Install lighttpd

Just enter the following command:
# make install
Start lighttpd:
# /etc/init.d/lighttpd start
Watch out for lighttpd log files for any problems:
# tail -f /var/log/messages
# tail -f /var/log/lighttpd/error.log
# tail -f /var/log/lighttpd/scripts.log
# tail -f /var/log/lighttpd/access.log

A note about binary package upgrade method

You can download rpm file or use yum / apt-get command:
apt-get update lighttpd
yum update lighttpd

Download of the day: Lighttpd web server 1.4.17

Posted on in Categories Download of the day, lighttpd, Linux, RedHat/Fedora Linux, UNIX, Windows server last updated September 1, 2007

Finally, new Lighttpd 1.4.17 has been released. This is security and bug fix update. I had updated my box and now this site is powered by 1.4.17.

Download lighttpd

Visit official web site here

Install and configure Lighttpd under Linux

(a) See how to install and configure lighttpd under RHEL

(b) See all lighttpd related articles

Display or list web server headers

Posted on in Categories Apache, CentOS, Debian Linux, Howto, lighttpd, Linux, UNIX, Windows server last updated August 16, 2007

If you are looking to debug a problem with HTTP server and would like to see HTTP requests / header responses in raw format, use shell tools or Firefox extensions.

Why do I need to dig out web server headers?

[a] Help debugging web application.

[b] See which kind of web server the remote site is using.

[c] See the cookies sent by remote site

[d] Learn http protocol and much more

Let us see Linux / UNIX command line tools to dig out headers.

wget command

wget command is not just downloading utility. It can be served as debugging tool. You need to pass -S option to wget command, which will print the headers sent by HTTP servers and responses sent by FTP servers.
$ wget -S http://theos.in
Output:

--01:32:15--  http://theos.in/
           => `index.html'
Resolving theos.in... 74.86.49.131
Connecting to theos.in|74.86.49.131|:80... connected.
HTTP request sent, awaiting response...
  HTTP/1.0 200 OK
  Connection: close
  X-Pingback: http://theos.in/xmlrpc.php
  Content-Type: text/html; charset="UTF-8"
  Date: Thu, 16 Aug 2007 20:02:16 GMT
  Server: lighttpd
Length: unspecified [text/html]

    [     <=>                                                                                                                              ] 44,214        32.56K/s

01:32:17 (32.50 KB/s) - `index.html' saved [44214]

If you are just intrested in headers pass –spider option to wget. With this option wget behave as a Web spider, which means that it will not download the pages, just check that they are there.
$ wget -S --spider http://theos.in/

Other command line tools

UNIX / Linux offers so many options. You can try out old good telnet command, lynx browser, curl command and other tools:
$ lynx -head -dump http://theos.in/
$ curl -I http://theos.in/

You can use telnet as follows:
$ telnet theos.in 80
Output:

HEAD /index.php HTTP/1.1

Firefox extension

If you are using Windows or just wanted to get information on fly, try firefox extension – LiveHTTPHeaders. It adds a ‘Headers’ tab in ‘View Page Info’ of a web page. You can also see a tool in the ‘Tools->Web Development’ menu to be able to display http headers in real time while pages are being downloaded from the Internet or web server. It also allows you edit request headers and replay an URL (good for testing web app security).

Download LiveHTTPHeaders

=> Visit project download page here

Other options

You can use specialized tools such as WireShark network protocol analyzer or tcpdump for the same purpose.

Lighttpd block wget useragent for specific urls

Posted on in Categories Ask nixCraft, lighttpd, Linux, Security, Tips, Tuning, UNIX last updated August 12, 2007

One of regular reader asks a question:

My website powered by Lighttpd web server. I’d like to block Wget useragent for entire my domain.com site except for /downloads/ url section. How do I configure lighttpd?

You need to use $HTTP filed useragent and url combination. Just open your lighttpd.conf file and append code as follows.

Lighttpd block useragent wget configuration

# vi /etc/lighttpd/lighttpd.conf
Append config directive as follows:

$HTTP["useragent"] =~ "Wget" {
        $HTTP["url"] !~ "^/download($|/)" {
                url.access-deny = ( "" )
        }
  }

Where,

  • $HTTP[“useragent”] : Match on useragent i.e. Wget
  • $HTTP[“url”] : Match on url section such as /download/*. If there are nested blocks, this must be the most inner block.
  • =~ : Perl style regular expression match
  • !~ : Perl style regular expression not match

Just restart the webserver, enter:
# /etc/init.d/lighttpd restart

Now user can run wget on http://domain.com/download/* urls but not on http://domain.com/file.html or http://domain.com/dir/file

PHP Log All Errors to a Log File to Get Detailed Information

Posted on in Categories Apache, Howto, lighttpd, Linux, php, Security, UNIX last updated July 29, 2007

PHP offers simple but effective solution to log all errors to a log fiie.
On all production web server you must turn off displaying error to end users via a web browser. Remember PHP gives out lots of information about path, database schema and all other sort of sensitive information. You are strongly advised to use error logging in place of error displaying on production web sites. The idea is quite simple only developer should able to see php error log.
Continue reading “PHP Log All Errors to a Log File to Get Detailed Information”

Lighttpd web server setup custom PHP.INI file for each user or domain

Posted on in Categories Howto, lighttpd, Linux, RedHat/Fedora Linux, Ubuntu Linux, UNIX last updated July 16, 2007

You can provide each user or domain its own php.ini file. There are two basic ways to provide each user a php.ini file:

a) Setup chrooted jail for each domain and user will get /etc/php.ini inside each jail
b) Setup individual fastcgi instance for each domain along with php.ini

Let us say you have two domains as follows

  1. theos.in php.ini location /home/lighttpd/theos.in/php.ini
  2. cyberciti.biz php.ini location /home/lighttpd/cyberciti.biz/php.ini
  3. /etc/php.ini – generic file for the rest of all domains

You need to add following directives to lighttpd.conf file:

$HTTP["host"]  =~ "(^|\.)theos\.in$" {
  server.document-root = "/home/lighttpd/theos.in/http/"
  accesslog.filename         = "/var/log/lighttpd/theos.in/access.log"
  server.error-handler-404 = "/index.php?error=404"
  fastcgi.server    = ( ".php" =>
        ((
                "bin-path" => "/usr/bin/php-cgi -c /home/lighttpd/theos.in/php.ini",
                "socket" => "/home/lighttpd/theos.in/php-cgi.socket",
        ))
)
}

$HTTP["host"]  =~ "(^|\.)cyberciti\.biz$" {
  server.document-root = "/home/lighttpd/cyberciti.biz/http/"
  accesslog.filename         = "/var/log/lighttpd/theos.in/access.log"
  server.error-handler-404 = "/index.php?error=404"
  fastcgi.server    = ( ".php" =>
        ((
                "bin-path" => "/usr/bin/php-cgi -c /home/lighttpd/cyberciti.biz/php.ini",
                "socket" => "/home/lighttpd/cyberciti.biz/php-cgi.socket",
        ))
)
}

Note option -c /path/to/my/custom/php.ini passed to /usr/bin/php-cgi. It will force php to Look for php.ini file in the directory path specified by us.

Now end users can modify php.ini as per requirements.

Pitfalls

  • Although a user can make changes to php.ini file, you still need to restart a web server using root or equivalent privileges
  • This may also open your box to new security issue such as wrong php.ini settings or user can load any custom php modules

You can apply same settings to Apache web server using jail or lighttpd fastcgi as a proxy.

Lighttpd run php from different host using mod_proxy / mod_fastcgi

Posted on in Categories High performance computing, Howto, lighttpd, Linux, UNIX last updated July 10, 2007

Lighttpd allows you to run php from different hosts. This is quite useful:

a] If you want to run php 4 locally and php 5 from remote host
b] Load balancing dynamic content
c] Added layer for security for chrooted jails etc

If you would like to run wikipedia / sf.net like site, you can use this technique. You can use mod_proxy or standard mod_fastcgi for this purpose.

How it works?

You need to use spawn-fcgi binary that spawns fastcgi processes. With spawn-fcgi you can bind php to particular port or unix-domain socket (little fast as compare to tcp port). It will take off some load from the webserver you have to control the FastCGI process by a external program like spawn-fcgi.

For example following command uses unix-domain to launch fastcgi process:
spawn-fcgi -s /tmp/php-fastcgi.sock -f /usr/bin/php-cgi -u lighttpd -g lighttpd -C 5 -P /var/run/spawn-fcgi.pid

This one bind itself to TCP port 8081
spawn-fcgi -p 8081 -a 192.168.1.10 -f /usr/bin/php-cgi -u lighttpd -g lighttpd -C 5 -P /var/run/spawn-fcgi-1.pid

Where,

  • -f {fcgiapp} filename of the fcgi-application, e.g php – /usr/bin/php-cgi
  • -a {addr} : bind to ip address
  • -p {port} : bind to tcp-port
  • -s {path} : bind to unix-domain socket
  • -C {childs} : (PHP only) numbers of childs to spawn (default 5)
  • -P {path} : name of PID-file for spawed process, so that we can kill process later on
  • -n : no fork (for daemontools)
  • -c {dir} : chroot to directory
  • -u {user} : change to user-id
  • -g {group} : change to group-id

Using mod_proxy / mod_fastcgi, we can process everything on 192.168.1.10 or cluster of php servers:

Web server <----> php-request <----> PHP listing on 192.168.1.10:8080 

A php / ruby / java app cluster server:

Web server <----> php-request <----> // PHP listing on 192.168.1.10:8080
                              // PHP listing on 192.168.1.11:8080
                             // PHP listing on 192.168.1.12:8080 

Task: Run php from 192.168.1.10 and 8081 port

Make sure you copy spawn-fcgi file to 192.168.1.10, now enter following command:
# spawn-fcgi -p 8081 -a 192.168.1.10 -f /usr/bin/php-cgi -u lighttpd -g lighttpd -C 10 -P /var/run/spawn-fcgi.pid
Make sure firewall is not blocking access to 192.168.1.10:8081

Now open ligttpd.conf on other host and enter mod_fastcgi as config as follows:

fastcgi.server = ( ".php" =>
   ((
       "host" => "192.168.1.10",
       "port" => 8081
   ))
)

Save and close the file. Restart lighttpd:
# /etc/init.d/lighttpd restart

You can use mod_proxy configuration as follows, if one of the hosts goes down the all requests for this one server are moved equally to the other servers.

$HTTP["host"] == "www.myweb2.0.com" {
  proxy.balance = "hash"
  proxy.server  = ( "" => ( ( "host" => "192.168.1.5","port" => 8080  ),
                            ( "host" => "192.168.1.6" ,"port" => 8080),
                            ( "host" => "192.168.1.7" ,"port" => 8080),
                            ( "host" => "192.168.1.8" ,"port" => 8080),
                            ( "host" => "192.168.1.9" ,"port" => 8080) ) )
}

This is just an introduction, feel free to explore mod_proxy documentation for more information.