Lighttpd Traffic Shaping: Throttle Connections Per Single IP (Rate Limit)

If you do not control or throttle end users, your server may run out of resources. Spammers, abuser and badly written bots can eat up all your bandwidth. A webserver must keep an eye on connections and limit connections per second. This is serving 101. The default is no limit. Lighttpd can limit the throughput for each single connection (per IP) or for all connections. You also need to a use firewall to limit connections per second. In this article I will cover firewall and lighttpd web server settings to throttle end users. The firewall settings can be applied to other web servers such as Apache / Nginx and IIS server behind PF / netfilter based firewall.

Lignttpd: Limit All Connections

You can limit the throughput for all connections to the given limit in kbyte/s. Open lighttpd.conf file:
# vi lighttpd.conf
Set limit to 1024 kbyte/s:
server.kbytes-per-second=1024
Save and close the file. Reload lighttpd server:
# service lighttpd reload

Lighttpd: Limit Throughput For Each Single Connection

Set limit to 64 kbyte/s for each single connection per IP:
connection.kbytes-per-second=64
Reload lighttpd server:
# service lighttpd reload

How Do I Set a Limit Only For Virtual Host?

You can set limit for virtual host only as follows (limit traffic to theos.in to 64 kbyte/s:

 
    $HTTP["host"] == "theos.in" {
      server.kbytes-per-second = 64
    }
 

How Do I Limit Connections Per Single IP?

You need to use a firewall such as *BSD PF or Linux netfilter firewall.

*BSD PF Firewall Example - Limit Connections Per Single IP

Add following rules to your /etc/pf.conf file. The following rules will protect the webserver against hosts making more than 100 connections in 10 seconds. Any IP which connects faster than this rate will have its address added to the table and have all states originating from it flushed. Any new packets from same IP to web server will be dropped:

webserver_ip="202.54.1.1"
table <abusive_ips> persist
block quick from <abusive_ips>
pass in on $ext_if proto tcp to $webserver_ip port www keep state (max-src-conn-rate 100/10, overload <bad_hosts> flush global)

Another example:

webserver_ip="202.54.1.1"
table <abusive_ips> persist
block in quick from <abusive_ips>
pass in on $ext_if proto tcp to $webserver_ip port www flags S/SA keep state (max-src-conn 100, max-src-conn-rate 15/5, overload <abusive_ips> flush)

Here is what it does:

• Limits the maximum number of connections per source to 100 (some browsers can open 30-40 connections per IP, so keep this to 100)
• Next, limit the number of connections per second or span of seconds. For e.g. rate limit the number of connections to 15 in a 5 second span.
• If anyone breaks our rules add them to our abusive_ips table and block them for making any further connections.
• Finally, flush keyword kills all states created by the matching rule which originate from the host which exceeds these limits.

Feel free to adjust settings as per your setup.

Linux Netfilter (Iptables) Examples To Limit Connections

The following example will drop incoming connections if IP make more than 10 connection attempts to port 80 within 100 seconds (add rules to your iptables shell script)

IPT=/sbin/iptables
# Max connection in seconds
SECONDS=100
# Max connections per IP
BLOCKCOUNT=10
# ....
# ..
# default action can be DROP or REJECT
DACTION="DROP"
$IPT -I INPUT -p tcp --dport 80 -i eth0 -m state --state NEW -m recent --set
$IPT -I INPUT -p tcp --dport 80 -i eth0 -m state --state NEW -m recent --update --seconds ${SECONDS} --hitcount ${BLOCKCOUNT} -j ${DACTION}
# ....
# ..

Again, feel free to adjust settings as per your setup.

Recommend Readings:

1. Sample PF firewall script.
2. Sample Iptables firewall script.
3. The official lighttpd documentation.
4. Iptables recent patch documentation.
5. The official pf documentation.

Featured Articles:

• 20 Linux System Monitoring Tools Every SysAdmin Should Know
• 20 Linux Server Hardening Security Tips
• 10 Greatest Open Source Software Of 2009
• My 10 UNIX Command Line Mistakes
• Top 5 Email Client For Linux, Mac OS X, and Windows Users
• Top 20 OpenSSH Server Best Security Practices
• Top 10 Open Source Web-Based Project Management Software
• Top 5 Linux Video Editor Software