≡ Menu

lighttpd

Reddit is a social news website (just like digg) on which users can post links to content on the web. Today (June 18, 2008), became an open source project. All of the code and libraries written for Reddit became freely available. Here's a quick list of the open source products that reddit is built and runs upon:

Debian, lighttpd, HAProxy, PostgreSQL, Slony-I, various python libraries, Psychopg, pylons, Solr, Tomcat, Ganglia, Mercurial, Git, gettext (translation), daemontools, and memcached

Today we're excited to announce that we're open sourcing reddit.

All reddit code is licensed under the Common Public Attribution License, which is basically the Mozilla license with a handful of changes. Specifically, the CPAL stipulates that when running reddit's code publicly, any changes to the code must be made available publicly and the site must make clear that it is running reddit code.

View Reddit code online here.

Many of our regular readers like to know more about lighttpd hotlink protection using mod_rewrite. Lighttpd can use HTTP referrer to detect hotlink and can be configured to partially protect hosted media from inline linking, usually by not serving the media or by serving a different file.

Lighttpd anti hotlinking configuration - redirect to another media

Open lighttpd.conf configuration file:
# vi /etc/lighttpd/lighttpd.conf
Append the following directive to redirect to a default picture called /hotlink.png:

$HTTP["referer"] =~ ".*BADDOMAIN\.com.*|.*IMAGESUCKERDOMAIN\.com.*|.*blogspot\.com.*" {
  url.rewrite = ("(?i)(/.*\.(jpe?g|png))$" => "/hotlink.png" )
}

So if anyone from *.blogspot.com linked www.cyberciti.biz/image.png it will be replaced with www.cyberciti.biz/hotlink.png. I've written small script to detect excessive hotlink from log file and ban all those domains. Most types of electronic media can be redirected this way, including video files, music files, and animations etc.

Related: Apache web server user can stop leechers using mod_rewrite / .htaccess rules.

AWStats is a free powerful tool that generates advanced web, streaming, ftp or mail server statistics, graphically. This log analyzer works as a CGI or from command line and shows you all possible information your log contains, in few graphical web pages. It uses a partial information file to be able to process large log files, often and quickly. It can analyze log files from all major server tools like Apache log files (NCSA combined/XLF/ELF log format or common/CLF log format), WebStar, IIS (W3C log format) and a lot of other web, proxy, wap, streaming servers, mail servers and some ftp servers.

You can easily configure awstats under Lighttpd web server.
[click to continue…]

Gzip is the most popular and effective compression method. Most modern web browser supports and accepts compressed data transfer. By gziping response time can reduced by 60-70% as compare to normal web page. The end result is faster web site experience for both dial up (they're not dead yet - I've dial up account for backup purpose) and broadband user. I've already written about speeding up Apache 2.x web access or downloads with mod_deflate.

mod_compress for Lighttpd 1.4.xx

Lighttpd 1.4.xx supports gzip compression using mod_compress. This module can reduces the network load and can improve the overall throughput of the webserver. All major http-clients support compression by announcing it in the Accept-Encoding header as follows:

Accept-Encoding: gzip, deflate

If lighttpd sees this header in the request, it can compress the response using one of the methods listed by the client. The web server notifies the web client of this via the Content-Encoding header in the response:

Content-Encoding: gzip

This is used to negotiate the most suitable compression method. Lighttpd support deflate, gzip and bzip2.

Configure mod_compress

Open your lighttpd.conf file:
# vi /etc/lighttpd/lighttpd.conf
Append mod_compress to server.modules directive:
server.modules += ( "mod_compress" )
Setup compress.cache-dir to stored all cached file:
compress.cache-dir = "/tmp/lighttpdcompress/"
Finally, define mimetypes to get compressed. Following will allow to compress javascript, plain text files, css file,xml file etc:

compress.filetype           = ("text/plain","text/css", "text/xml", "text/javascript" )

Save and close the file. Create /tmp/lighttpdcompress/ file:
# mkdir -p /tmp/lighttpdcompress/
# chown lighttpd:lighttpd /tmp/lighttpdcompress/

Restart lighttpd:
# /etc/init.d/lighttpd restart

How do I enable mod_compress per virtual host?

Use conditional $HTTP host directive, for example turn on compression for theos.in:

$HTTP["host"] =~ "theos\.in" {
  compress.cache-dir = "/var/www/cache/theos.in/"
}

PHP dynamic compression

Open php.in file:
# vi /etc/php.ini
To compress dynamic content with PHP please enable following two directives:
zlib.output_compression = On
zlib.output_handler = On

Save and close the file. Restart lighttpd:
# service lighttpd restart

Cleaning cache directory

You need to run a shell script for cleaning out cache directory.

See also:

Lighttpd web server will generate a directory listing if a directory is requested and no index-file was found in that directory. mod_dirlisting is one of the modules that is loaded by default and doesn't have to be specified on server.modules to work.

Task: Enable Directory Listings Globally

Open lighttpd configuration file:
# vi /etc/lighttpd/lighttpd.conf
Append / modify
server.dir-listing = "enable"
OR
dir-listing.activate = "enable"
Save and close the file. Restart lighttpd:
# /etc/init.d/lighttpd restart
To disable directory listing, use:
dir-listing.activate = "disable"

Enable directory listing only for a directory

You can also enable or disable listing on selected url / directory combination. For example, display directory listing only for /files/:
$HTTP["url"] =~ "^/files($|/)" { server.dir-listing = "enable" }
OR
$HTTP["url"] =~ "^/files($|/)" { dir-listing.activate = "enable" }

Further readings:

Lighttpd / Apache : Run Xcache in Chrooted Jail

Recently I wrote about installing and running Xcache under Red hat enterprise Linux and CentOS Linux. By default Xcache use /dev/zero for caching. All you have to do is create /dev/zero in chrooted jail. Type the following command (assuming that your jail is located at /lighttpd.jail directory):
# mkdir -p /lighttpd.jail/dev
# mknod -m 666 /lighttpd.jail/dev/zero c 1 5

Just restart your web server and xcache should work under chrooted lighttpd web server.

Many people asked me to write about setting up Lighttpd under CentOS or RHEL 5 Linux using chroot() call. The instructions are almost same but you need to make little modification as compare to Debian / Ubuntu Linux instructions.

For example purpose we will build jail at /webroot location.
=> Default document root : /home/lighttpd/default/
=> Port : 80
=> IP: Your Public IP address
=> Virtual domain1: /home/lighttpd/vdomain1.com/
=> Virtual domain1 access log file: /var/log/lighttpd/vomain1.com/
=> Default access log file:/var/log/lighttpd/access.log
=> Default error log file:/var/log/lighttpd/error.log
=> Default php error log file: /var/log/lighttpd/php.log

Assumptions

These installation instructions assume you have:

  • Linux distribution
  • Required RPMs (see below for installation instructions)
    • php, php-pear, php-common, php-pdo, php-ldap, php-gd, php-cli, php-mysql
    • mysql, mysql-server etc
    • lighttpd, lighttpd-fastcgi (rpm available here)
  • Installations were tested on Red Hat Enterprise Linux v4/5 or CentOS v4/5 or Fedora Linux 7

Step # 1: Install required packages

Install php and related packages:
# yum install php php-pear php-common php-pdo php-ldap php-gd php-cli php-mysql
Install mysql and related packages:
# yum install mysql mysql-server
Install lighttpd and mod_fastcgi for lighttpd:
# rpm -ivh http://dag.wieers.com/rpm/packages/lighttpd/lighttpd-1.4.18-1.el5.rf.i386.rpm
# rpm -ivh http://dag.wieers.com/rpm/packages/lighttpd/lighttpd-fastcgi-1.4.18-1.el5.rf.i386.rpm

Step # 2: Create /webroot and related directories

# mkdir /webroot
# cd /webroot
# mkdir etc
# mkdir tmp
# chmod 1777 tmp/
# mkdir -p usr/bin
# mkdir -p home/lighttpd/default
# mkdir -p var/run/lighttpd
# mkdir -p var/log/lighttpd
# chown lighttpd:lighttpd var/run/lighttpd/
# chown lighttpd:lighttpd var/log/lighttpd/
# chown -R lighttpd:lighttpd home/

Step # 3: Install chroot script

You need to download and install my script that will help you to build lighttpd in jail:
# cd /sbin/
# wget http://www.cyberciti.biz/files/lighttpd/l2chroot.txt
# mv l2chroot.txt l2chroot
# chmod +x l2chroot

Step # 4: Install php in jail

Now copy php-cgi binary and related shared libraries using l2chroot script:
# cd /webroot/usr/bin
# cp /usr/bin/php-cgi .
# l2chroot php-cgi

Step # 5: Copy required files to /etc

Now you must copy php.ini and related all files to /etc/
# cd /webroot/etc
# cp /etc/passwd .
# cp /etc/group .
# cp /etc/hosts .
# cp /etc/nsswitch.conf .
# cp /etc/resolv.conf .
# cp /etc/php.ini .
# cp -avr /etc/php.d/ .
# cp -avr /etc/ld* .

Update (Oct-1-2008, 1:52pm) : You need to copy entire /etc/ and /usr/share/zoneinfo files to work with latest php version:
# cd /webroot/etc
# /bin/cp -avr /etc/* .

Copy all files from /usr/share/zoneinfo/:
# mkdir -p /webroot/usr/share/
# cd /webroot/usr/share/
# cp -avr /usr/share/zoneinfo/ .

Open group and passwd file and only keep entries for root and lighttpd user:
# vi /webroot/etc/group
Make sure file look as follows:
root:x:0:root
lighttpd:x:101:

Also open passwd file inside jail:
# vi /webroot/etc/passwd
Make sure file look as follows:
root:x:0:0:root:/root:/bin/bash
lighttpd:x:100:101:lighttpd web server:/srv/www/lighttpd:/sbin/nologin

Step # 5: Copy php modules

Now copy php mysql support, php gd and other all modules:
# cd /webroot/usr/lib/
# cp -avr /usr/lib/php/ .
# cd php/modules
# for l in *.so; do l2chroot $l; done

Step # 6: Configure lighttpd chroot call

Open /etc/lighttpd/lighttpd.conf file:
# vi /etc/lighttpd/lighttpd.conf
Setup default document root and chroot directory:
server.document-root = "/home/lighttpd/default/"
server.chroot="/webroot"

Save and close the file.

Step # 7: Restart lighttpd

Type the following command:
# /etc/init.d/lighttpd restart

Jail size

# du -ch /webroot/
Output:

12K     /webroot/var/log/lighttpd
16K     /webroot/var/log
4.0K    /webroot/var/run/lighttpd
8.0K    /webroot/var/run
28K     /webroot/var
8.0K    /webroot/etc/ld.so.conf.d
36K     /webroot/etc/php.d
160K    /webroot/etc
8.0K    /webroot/home/lighttpd/default
12K     /webroot/home/lighttpd
16K     /webroot/home
5.3M    /webroot/lib
4.0K    /webroot/tmp
872K    /webroot/usr/lib/sse2
1.4M    /webroot/usr/lib/mysql
676K    /webroot/usr/lib/php/modules
4.0K    /webroot/usr/lib/php/pear
684K    /webroot/usr/lib/php
9.9M    /webroot/usr/lib
2.9M    /webroot/usr/bin
13M     /webroot/usr
19M     /webroot/
19M     total

Troubleshooting

Always go thought /var/log/messages and server log files:
# tail -f /var/log/messages

Download mysql testing script

Copy and test php mysql connectivity with this script.