≡ Menu

Lighttpd Traffic Shaping: Throttle Connections Per Single IP (Rate Limit)

If you do not control or throttle end users, your server may run out of resources. Spammers, abuser and badly written bots can eat up all your bandwidth. A webserver must keep an eye on connections and limit connections per second. This is serving 101. The default is no limit. Lighttpd can limit the throughput for each single connection (per IP) or for all connections. You also need to a use firewall to limit connections per second. In this article I will cover firewall and lighttpd web server settings to throttle end users. The firewall settings can be applied to other web servers such as Apache / Nginx and IIS server behind PF / netfilter based firewall.

Lignttpd: Limit All Connections

You can limit the throughput for all connections to the given limit in kbyte/s. Open lighttpd.conf file:
# vi lighttpd.conf
Set limit to 1024 kbyte/s:
server.kbytes-per-second=1024
Save and close the file. Reload lighttpd server:
# service lighttpd reload

Lighttpd: Limit Throughput For Each Single Connection

Set limit to 64 kbyte/s for each single connection per IP:
connection.kbytes-per-second=64
Reload lighttpd server:
# service lighttpd reload

How Do I Set a Limit Only For Virtual Host?

You can set limit for virtual host only as follows (limit traffic to theos.in to 64 kbyte/s:

 
    $HTTP["host"] == "theos.in" {
      server.kbytes-per-second = 64
    }
 

How Do I Limit Connections Per Single IP?

You need to use a firewall such as *BSD PF or Linux netfilter firewall.

*BSD PF Firewall Example - Limit Connections Per Single IP

Add following rules to your /etc/pf.conf file. The following rules will protect the webserver against hosts making more than 100 connections in 10 seconds. Any IP which connects faster than this rate will have its address added to the table and have all states originating from it flushed. Any new packets from same IP to web server will be dropped:

webserver_ip="202.54.1.1"
table <abusive_ips> persist
block quick from <abusive_ips>
pass in on $ext_if proto tcp to $webserver_ip port www keep state (max-src-conn-rate 100/10, overload <bad_hosts> flush global)

Another example:

webserver_ip="202.54.1.1"
table <abusive_ips> persist
block in quick from <abusive_ips>
pass in on $ext_if proto tcp to $webserver_ip port www flags S/SA keep state (max-src-conn 100, max-src-conn-rate 15/5, overload <abusive_ips> flush)

Here is what it does:

  • Limits the maximum number of connections per source to 100 (some browsers can open 30-40 connections per IP, so keep this to 100)
  • Next, limit the number of connections per second or span of seconds. For e.g. rate limit the number of connections to 15 in a 5 second span.
  • If anyone breaks our rules add them to our abusive_ips table and block them for making any further connections.
  • Finally, flush keyword kills all states created by the matching rule which originate from the host which exceeds these limits.

Feel free to adjust settings as per your setup.

Linux Netfilter (Iptables) Examples To Limit Connections

The following example will drop incoming connections if IP make more than 10 connection attempts to port 80 within 100 seconds (add rules to your iptables shell script)

IPT=/sbin/iptables
# Max connection in seconds
SECONDS=100
# Max connections per IP
BLOCKCOUNT=10
# ....
# ..
# default action can be DROP or REJECT 
DACTION="DROP"
$IPT -I INPUT -p tcp --dport 80 -i eth0 -m state --state NEW -m recent --set
$IPT -I INPUT -p tcp --dport 80 -i eth0 -m state --state NEW -m recent --update --seconds ${SECONDS} --hitcount ${BLOCKCOUNT} -j ${DACTION}
# ....
# ..

Again, feel free to adjust settings as per your setup.

Recommend Readings:

  1. Sample PF firewall script.
  2. Sample Iptables firewall script.
  3. The official lighttpd documentation.
  4. Iptables recent patch documentation.
  5. The official pf documentation.
Tweet itFacebook itGoogle+ itPDF itFound an error/typo on this page?

{ 15 comments… add one }

  • Vasi June 22, 2009, 3:13 am

    Excellent information thanks a lot

  • Swapnil K. Chaudhari June 22, 2009, 6:08 am

    Hi,
    I have a query regardig Limiting Connections Per Single IP.

    Cosider a scenario of a big company, where there is only one public IP address available and all the users use DHCP or private IP. So if say 100 emplyee in the company access the same web site simultaneously (For checking mails, or some other common information). Web server will see that same public IP is generating 100 connections and put the public IP of company in black list, and employees won’t be able to connect to that web server.

    How can we avoid this scenario?

    Thanks,
    Swapnil.

    • nixCraft June 22, 2009, 8:36 am

      You can always white list certain IPs / subnets or create class of IPs to exclude throttling.

  • kunal June 24, 2009, 4:47 am

    How can i throttle connections per single IP in apache. Earlier there used to be a module named as mod_evassive but now i dont think its available any more.


    Kunal

  • nixCraft June 24, 2009, 5:20 am

    Try,
    mod_bandwidth:

    “Mod_bandwidth” is a module for the Apache webserver that enable the setting of server-wide or per connection bandwidth limits, based on the directory, size of files and remote IP/domain.

    Download : http://www.cohprog.com/mod_bandwidth.html
    mod_limitipconn:

    It allows web server administrators to limit the number of simultaneous downloads permitted from a single IP address.

    Download : http://dominia.org/djao/limitipconn2.html

  • kunal June 25, 2009, 2:58 am

    Thanks Vivek,

    Will definitely give a try to this.


    Kunal

  • geeth August 3, 2009, 12:14 pm

    The blog is really superb stuff, could you please mention the regarding apache module too

  • Vicent Gonzalez i Castells August 12, 2009, 7:01 am

    Great job. A good information.

    Following Swapnil comment, it’s a great problem to create a white list. We can think about to create a white list of universitues IP’s of the world. For me, it’s a good technique to apply into small environments like a enterprise network.

    Santi

  • The Mikeness July 21, 2010, 9:55 pm

    Note that limiting concurrent connections per IP to 100 may block some organizations that use a shoddy HTTP Proxy, as it will usually refuse to hold persistent connections, and there may be dozens or hundreds of users behind the proxy. What the end users will begin to see if you do alot of this sort of filtering without using extreme care is things failing at odd places, too many requests within a few seconds randomly causing some portions of pages to not load completely (like CSS and image files), AJAX requests to fail, etc.

    I just though I’d mention that incase this sort of thinking becomes prominent among people running webservers and big sites start to have hard to troubleshoot problems that may end up ruining your reputation amongst your users for being unable to operate a reliable service.

    • nixCraft June 12, 2011, 9:38 am

      You serve all static content (css/js/images/videos) from other domain or cdn to avoid this issue.

  • gadelkareem August 28, 2010, 2:49 pm

    What about evasive.max-conns-per-ip = 20 for lighty?

  • dirk March 18, 2011, 12:11 pm

    evasive.max-conns-per-ip = 14
    is a good value for lighttpd. no need to install firewalls.

  • David September 18, 2011, 8:49 pm

    Excelent, but is there a way you can do the same using apache instead of lighthttp?

    • JC October 22, 2011, 8:44 am

      There is mod_evasive for Apache.

Leave a Comment