≡ Menu


Lighttpd Drupal CMS Clean URL ( SEO ) Rules Set Configuration

Drupal is modular framework and content management system (CMS) and works under Lighttpd too. By default, Drupal passes path arguments to itself via its internally generated URLs. This results in URLs that look like the following: "http://www.example.com/?q=node/83." This can make URLs hard to read and it also stops many search engines, like Google, from indexing the pages with these URLs.

You can tell Drupal to use "clean URLs", eliminating the "?q=" in internal URLs. Assuming that your site hosted in rootdirectory itself, open your lighttpd.conf file or domain configuration file:
# vi lighttpd.conf
Make sure mod_rewrite is enabled:
server.modules += ( "mod_rewrite" )
Append following configuration directives:

url.rewrite-final = (
  "^/system/test/(.*)$" => "/index.php?q=system/test/$1",
  "^/system/test-clean-url/(.*)$" => "/index.php?q=system/test-clean-url/$1",
  "/rss.xml$" => "/index.php?q=rss.xml",
  "^/search/(.*)$" => "/index.php?q=search/$1",
  "^/([^.?]*)\?(.*)$" => "/index.php?q=$1&$2",
  "^/([^.?]*)$" => "/index.php?q=$1",
# Error 404
server.error-handler-404  = "/index.php"
# stop these
url.access-deny = ( "~", ".inc", ".engine", ".install", ".module", ".sh", "sql", ".theme", ".tpl.php", ".xtmpl", "Entries", "Repository", "Root" )

Save and close the file. Restart lighttpd, enter:
# /etc/init.d/lighttpd restart
Feel free to adjust rules as per your setup.

Further readings:

Debian Linux Security Update: Lighttpd DoS and Gaim Package Remote Security Issues

Debian Linux project released today bug fixes for lighttpd and gaim package.

Gaim packages fix execution of arbitrary code

It was discovered that gaim, an multi-protocol instant messaging client, was vulnerable to several integer overflows in its MSN protocol handlers. These could allow a remote attacker to execute arbitrary code.

lighttpd packages fix multiple DOS issues

Several local/remote vulnerabilities have been discovered in lighttpd, a fast webserver with minimal memory footprint.

a) lighttpd 1.4.18, and possibly other versions before 1.5.0, does not properly calculate the size of a file descriptor array, which allows remote attackers to cause a denial of service (crash) via a large number of connections, which triggers an out-of-bounds access.

b) connections.c in lighttpd before 1.4.16 might accept more connections than the configured maximum, which allows remote attackers to cause a denial of service (failed assertion) via a large number of connection attempts.

How do I fix lighttpd and gaim security issues?

First, update the internal database, enter:
# apt-get update
Install corrected packages, enter:
# apt-get upgrade

Apache Web Server: Log Analysis and Server Status Monitoring Tool

wtop is really cool application for web server log analysis and to see server stats at a glance. It also has powerful log grepping capability. It is just like 'top' for your webserver.

It can find out number of searches or signups per seconds. It can also create histogram of response time. There is also another tool called logrep a powerful command-line program for ad-hoc analysis and filtering for log files. You can dig up lots of information using wtop tools.

You need Python version 2.5 to run wtop.

Download wtop

Type the following command:
$ cd /tmp
$ wget http://wtop.googlecode.com/files/wtop-0.5.6.tar.gz
$ tar -zxvf wtop-0.5.6.tar.gz
$ cd wtop-0.5.6
# python setup.py install

Configuring wtop

Once installed you can start using the tool immediately. You need to edit /etc/wtop.cfg file to setup parameters, Apache log files and other directives
# vi /etc/wtop.cfg
Sample configuration file:

# This must match your webserver log format. You MUST have at least %h, %r and %D
LOG_FORMAT=%h %l %u %t "%r" %>s %B "%{Referer}i" "%{User-Agent}i" %D
# max time before a request is logged in the "slow" column
# minimum requests/second before a URL class appears in top mode
# you can extend these to make any classes you wish
# the generic pattern is applied if a line does not match any
# of the named classes. By default it uses the top-level directory.
# incomplete list of known web robots
robots = r'(?:nutch|MSRBOT|translate.google.com|Feedster|Nutch|Gaisbot|Snapbot|VisBot|libwww|CazoodleBot|polybot|VadixBot|Sogou|SBider|BecomeBot|Yandex|Pagebull|chudo|Pockey|nicebot|entireweb|FeedwhipBOT|ConveraCrawler|NG/2.0|WebImages|Factbot|information-online|gsa-crawler|Jyxobot|SentinelCrawler|BlogPulseLive|YahooFeedSeeker|GurujiBot|wwwster|Y\!J-SRD|Findexa|SurveyBot|yetibot|discoveryengine|fastsearch|noxtrum|Googlebot|Snapbot|OGSearchSpider|heritrix|nutch-agent|Slurp|msnbot|cuill|Mediapartners|YahooSeeker|GrabPERF|keywen|ia_archiver|crawler.archive.org|Baiduspider|larbin|shopwiki)'

Now simply type wtop at a shell prompt:
$ wtop$
See all human traffic, enter:
$ logrep -m top -h access.log
See response times for all MSNBot homepage hits:
$ logrep -m grep -g MSNBot -i home -o status,msec,url access.log
Display the current log for traffic to pages about wordpress or themes sent from google.com
$ logrep -m tail --f 'url~wordpress|themes,ref~google.com' access.log

Further readings:

Lighttpd mod_rewrite Hotlink Protection To Display Image Message

Many of our regular readers like to know more about lighttpd hotlink protection using mod_rewrite. Lighttpd can use HTTP referrer to detect hotlink and can be configured to partially protect hosted media from inline linking, usually by not serving the media or by serving a different file.

Lighttpd anti hotlinking configuration - redirect to another media

Open lighttpd.conf configuration file:
# vi /etc/lighttpd/lighttpd.conf
Append the following directive to redirect to a default picture called /hotlink.png:

$HTTP["referer"] =~ ".*BADDOMAIN\.com.*|.*IMAGESUCKERDOMAIN\.com.*|.*blogspot\.com.*" {
  url.rewrite = ("(?i)(/.*\.(jpe?g|png))$" => "/hotlink.png" )

So if anyone from *.blogspot.com linked www.cyberciti.biz/image.png it will be replaced with www.cyberciti.biz/hotlink.png. I've written small script to detect excessive hotlink from log file and ban all those domains. Most types of electronic media can be redirected this way, including video files, music files, and animations etc.

Related: Apache web server user can stop leechers using mod_rewrite / .htaccess rules.

Linux: Install Django Open Source Framework

Django is a high-level Python Web framework (open source framework) that encourages rapid development and clean, pragmatic design. Django is awesome programming framework. Red hat magazine has published excellent tutorial:

In today's world, web development is all about turnaround. Businesses want to maximize production outcome while minimizing development and production time. Small, lean development teams are increasingly becoming the normal large development departments. Enter Django: a popular Python web framework that invokes the RWAD (rapid web application development) and DRY (don't repeat yourself) principles with clean, pragmatic design.

This article is not about teaching you how to program in Python, nor how to use the Django framework. It's about showing how to promote your Django applications onto an existing Apache or Lighttpd environment.

=> Installing/Configuring/Caching Django on your Linux server

Lighttpd Install and Configure AwStats Software Log Analyzer

AWStats is a free powerful tool that generates advanced web, streaming, ftp or mail server statistics, graphically. This log analyzer works as a CGI or from command line and shows you all possible information your log contains, in few graphical web pages. It uses a partial information file to be able to process large log files, often and quickly. It can analyze log files from all major server tools like Apache log files (NCSA combined/XLF/ELF log format or common/CLF log format), WebStar, IIS (W3C log format) and a lot of other web, proxy, wap, streaming servers, mail servers and some ftp servers.

You can easily configure awstats under Lighttpd web server.
[click to continue…]

mod_compress: Lighttpd Gzip Compression To Improve Download and Browsing Speed

Gzip is the most popular and effective compression method. Most modern web browser supports and accepts compressed data transfer. By gziping response time can reduced by 60-70% as compare to normal web page. The end result is faster web site experience for both dial up (they're not dead yet - I've dial up account for backup purpose) and broadband user. I've already written about speeding up Apache 2.x web access or downloads with mod_deflate.

mod_compress for Lighttpd 1.4.xx

Lighttpd 1.4.xx supports gzip compression using mod_compress. This module can reduces the network load and can improve the overall throughput of the webserver. All major http-clients support compression by announcing it in the Accept-Encoding header as follows:

Accept-Encoding: gzip, deflate

If lighttpd sees this header in the request, it can compress the response using one of the methods listed by the client. The web server notifies the web client of this via the Content-Encoding header in the response:

Content-Encoding: gzip

This is used to negotiate the most suitable compression method. Lighttpd support deflate, gzip and bzip2.

Configure mod_compress

Open your lighttpd.conf file:
# vi /etc/lighttpd/lighttpd.conf
Append mod_compress to server.modules directive:
server.modules += ( "mod_compress" )
Setup compress.cache-dir to stored all cached file:
compress.cache-dir = "/tmp/lighttpdcompress/"
Finally, define mimetypes to get compressed. Following will allow to compress javascript, plain text files, css file,xml file etc:

compress.filetype           = ("text/plain","text/css", "text/xml", "text/javascript" )

Save and close the file. Create /tmp/lighttpdcompress/ file:
# mkdir -p /tmp/lighttpdcompress/
# chown lighttpd:lighttpd /tmp/lighttpdcompress/

Restart lighttpd:
# /etc/init.d/lighttpd restart

How do I enable mod_compress per virtual host?

Use conditional $HTTP host directive, for example turn on compression for theos.in:

$HTTP["host"] =~ "theos\.in" {
  compress.cache-dir = "/var/www/cache/theos.in/"

PHP dynamic compression

Open php.in file:
# vi /etc/php.ini
To compress dynamic content with PHP please enable following two directives:
zlib.output_compression = On
zlib.output_handler = On

Save and close the file. Restart lighttpd:
# service lighttpd restart

Cleaning cache directory

You need to run a shell script for cleaning out cache directory.

See also: