≡ Menu

daemontools

Reddit is a social news website (just like digg) on which users can post links to content on the web. Today (June 18, 2008), became an open source project. All of the code and libraries written for Reddit became freely available. Here's a quick list of the open source products that reddit is built and runs upon:

Debian, lighttpd, HAProxy, PostgreSQL, Slony-I, various python libraries, Psychopg, pylons, Solr, Tomcat, Ganglia, Mercurial, Git, gettext (translation), daemontools, and memcached

Today we're excited to announce that we're open sourcing reddit.

All reddit code is licensed under the Common Public Attribution License, which is basically the Mozilla license with a handful of changes. Specifically, the CPAL stipulates that when running reddit's code publicly, any changes to the code must be made available publicly and the site must make clear that it is running reddit code.

View Reddit code online here.

Lighttpd allows you to run php from different hosts. This is quite useful:

a] If you want to run php 4 locally and php 5 from remote host
b] Load balancing dynamic content
c] Added layer for security for chrooted jails etc

If you would like to run wikipedia / sf.net like site, you can use this technique. You can use mod_proxy or standard mod_fastcgi for this purpose.

How it works?

You need to use spawn-fcgi binary that spawns fastcgi processes. With spawn-fcgi you can bind php to particular port or unix-domain socket (little fast as compare to tcp port). It will take off some load from the webserver you have to control the FastCGI process by a external program like spawn-fcgi.

For example following command uses unix-domain to launch fastcgi process:
spawn-fcgi -s /tmp/php-fastcgi.sock -f /usr/bin/php-cgi -u lighttpd -g lighttpd -C 5 -P /var/run/spawn-fcgi.pid

This one bind itself to TCP port 8081
spawn-fcgi -p 8081 -a 192.168.1.10 -f /usr/bin/php-cgi -u lighttpd -g lighttpd -C 5 -P /var/run/spawn-fcgi-1.pid

Where,

  • -f {fcgiapp} filename of the fcgi-application, e.g php - /usr/bin/php-cgi
  • -a {addr} : bind to ip address
  • -p {port} : bind to tcp-port
  • -s {path} : bind to unix-domain socket
  • -C {childs} : (PHP only) numbers of childs to spawn (default 5)
  • -P {path} : name of PID-file for spawed process, so that we can kill process later on
  • -n : no fork (for daemontools)
  • -c {dir} : chroot to directory
  • -u {user} : change to user-id
  • -g {group} : change to group-id

Using mod_proxy / mod_fastcgi, we can process everything on 192.168.1.10 or cluster of php servers:

Web server <----> php-request <----> PHP listing on 192.168.1.10:8080 

A php / ruby / java app cluster server:

Web server <----> php-request <----> // PHP listing on 192.168.1.10:8080
                              // PHP listing on 192.168.1.11:8080
                             // PHP listing on 192.168.1.12:8080 

Task: Run php from 192.168.1.10 and 8081 port

Make sure you copy spawn-fcgi file to 192.168.1.10, now enter following command:
# spawn-fcgi -p 8081 -a 192.168.1.10 -f /usr/bin/php-cgi -u lighttpd -g lighttpd -C 10 -P /var/run/spawn-fcgi.pid
Make sure firewall is not blocking access to 192.168.1.10:8081

Now open ligttpd.conf on other host and enter mod_fastcgi as config as follows:

fastcgi.server = ( ".php" =>
   ((
       "host" => "192.168.1.10",
       "port" => 8081
   ))
)

Save and close the file. Restart lighttpd:
# /etc/init.d/lighttpd restart

You can use mod_proxy configuration as follows, if one of the hosts goes down the all requests for this one server are moved equally to the other servers.

$HTTP["host"] == "www.myweb2.0.com" {
  proxy.balance = "hash"
  proxy.server  = ( "" => ( ( "host" => "192.168.1.5","port" => 8080  ),
                            ( "host" => "192.168.1.6" ,"port" => 8080),
                            ( "host" => "192.168.1.7" ,"port" => 8080),
                            ( "host" => "192.168.1.8" ,"port" => 8080),
                            ( "host" => "192.168.1.9" ,"port" => 8080) ) )
}

This is just an introduction, feel free to explore mod_proxy documentation for more information.