Lighttpd Traffic Shaping: Throttle Connections Per Single IP (Rate Limit)

Posted on in Categories Apache, CentOS, fedora linux, FreeBSD, Howto, Iptables, lighttpd, Linux, Networking, PF Firewall, RedHat/Fedora Linux, Security, Ubuntu Linux, UNIX last updated June 21, 2009

If you do not control or throttle end users, your server may run out of resources. Spammers, abuser and badly written bots can eat up all your bandwidth. A webserver must keep an eye on connections and limit connections per second. This is serving 101. The default is no limit. Lighttpd can limit the throughput for each single connection (per IP) or for all connections. You also need to a use firewall to limit connections per second. In this article I will cover firewall and lighttpd web server settings to throttle end users. The firewall settings can be applied to other web servers such as Apache / Nginx and IIS server behind PF / netfilter based firewall.

Comparison: Linux Disk Scheduler

Posted on in Categories Hardware, Links, Linux, News last updated August 9, 2008

I’ve already written about changing the I/O scheduler for hard disk under Linux and avoiding sudden outburst of disk I/O using ionice utility. Google has sponsored [email protected] to take a close look at the disk schedulers in Linux, particularly when combined with RAID. They have now published their findings.

mod_compress: Lighttpd Gzip Compression To Improve Download and Browsing Speed

Posted on in Categories Apache, High performance computing, Howto, lighttpd, Linux, News, php, UNIX last updated April 26, 2008

Gzip compression reduces response times by reducing the size of the HTTP response. This document describes gzipping http traffic which can reduces the response size by about 70%. Approximately 90% of today’s Internet traffic travels through browsers that claim to support compression.

EKA – The world's 4th Fastest Super Computer

Posted on in Categories Hardware, High performance computing, India, Linux, Linux distribution last updated March 29, 2008

In November 2007 EKA was announced, which is privately funded supercomputer. Eka, built by CRL, Pune is the world’s 4th fastest supercomputer, and the fastest one that didn’t use government funding. This is the same supercomputer referenced in Yahoo!’s recent announcement about cloud computing research at the Hadoop Summit. This article describes some of the technical details of Eka’s design and implementation.

My town – pune is home to some of the best supercomputing research and development facility into the area of software and other related technology. C-DAC (a scientific society of the Department of Information Technology, Government of India) has developed and supplied a range of high performance parallel computers, known as the PARAM series of supercomputers. Now CRL – a tata group company build the EKa one of the top 10 supercomputers in the world, Eka is the only system that was fully privately funded. All other systems used government money, so all of them are for captive use. This means that Eka is the only system in the top 10 that is available for commercial use without strings attached.

The computing infrastructure itself consists of:

  1. 1800 blades, 4 cores each. 3Ghz for each core.
  2. HP SFS clusters
  3. 28TB memory
  4. 80TB storage. Simple SATA disks. 5.2Gbps throughput.
  5. Lustre distributed file-system
  6. 20Gbps infiniband DDR. Eka was on the cutting edge of Infiniband technology. They sourced their infiniband hardware from an Israeli company and where amongst the first users of their releases – including beta, and even alpha quality stuff.
  7. Multiple Gigabit ethernets
  8. Linux is the underlying OS. Any Linux will work – RedHat, SuSe, your favorite distribution.

=> Building EKA – The world’s fastest privately funded supercomputer

Linux Fibre Channel over Ethernet implementation code released

Posted on in Categories Linux, Linux Scalability, Linux Virtualization, Networking, Open source coding, Storage last updated December 18, 2007

Intel has just released source code for Fibre Channel over Ethernet (FCoE). It provides some Fibre Channel protocol processing as well as the encapsulation of FC frames within Ethernet packets. FCoE will allow systems with an Ethernet adapter and a Fibre Channel Forwarder to login to a Fibre Channel fabric (the FCF is a “gateway” that bridges the LAN and the SAN). That fabric login was previously reserved exclusively for Fibre Channel HBAs. This technology reduces complexity in the data center by aiding network convergence. It is targeted for 10Gps Ethernet NICs but will work on any Ethernet NIC supporting pause frames. Intel will provide a Fibre Channel protocol processing module as well as an Ethernet based transport module. The Open-FC module acts as a LLD for SCSI and the Open-FCoE transport uses net_device to send and receive packets.

This is good news. I think one can compare bandwidth and throughput for copper and fiber Ethernet. If you are going to use copper you need to stay within 15m of the switch. This solution will try to bring down cost. One can connect to 8-10 server to central database server with 10G and there could be few more applications.

=> Open FCoE project home page