Linux Commodity computing is computing done on commodity computers as opposed to supermicrocomputers or boutique computers. Commodity computers are computer systems manufactured by multiple vendors, incorporating components based on open standards.
I’m not surprised at all. Linux runs on tiny phone to large server systems. According to IDC researchers (prediction) – spending on the Linux ecosystem will rise from $21 billion in 2007 to more than $49 billion in 2011, driven by rising enterprise deployments of Linux server operating systems.
Linux server deployments are expanding from infrastructure-oriented applications to more commercially oriented database and enterprise resource-planning workloads “that historically have been the domain of Microsoft Windows and Unix,” noted IDC analysts in a white paper commissioned by the nonprofit Linux Foundation.
“The early adoption of Linux was dominated by infrastructure-oriented workloads, often taking over those workloads from an aging Unix server or Windows NT 4.0 server that was being replaced,” according to the report’s authors, Al Gillen, Elaina Stergiades and Brett Waldman. These days, however, Linux is increasingly being “viewed as a solution for wider and more critical business deployments.”
=> Linux Ecosystem Spending To Exceed $49 Billion
I’ve worked with a various load balancing systems (LBS). They are complex pieces of hardware and software. In this post I will highlight some of the open source load balancing software. But what is load balancing?
It is nothing but a technique used to share (spared) load / services between two or more servers. For example, busy e-commerce or bank website uses load balancer to increase reliability, throughput, uptime, response time and better resource utilization. You can use following softwares as an advanced load balancing solution for web, cache, dns, mail, ftp, auth servers, VoIP services etc.
Linux Virtual Server (LVS)
LVS is ultimate open source Linux load sharing and balancing software. You can easily build a high-performance and highly available server for Linux using this software. From the project page:
Virtual server is a highly scalable and highly available server built on a cluster of real servers. The architecture of server cluster is fully transparent to end users, and the users interact with the cluster system as if it were only a single high-performance virtual server.
=> Project Web Site
Red Hat Cluster Suite
It is a high availability cluster software implementation from Linux leader Red Hat. It provide two type services:
- Application / Service Failover – Create n-node server clusters for failover of key applications and services
- IP Load Balancing – Load balance incoming IP network requests across a farm of servers
=> Product web page
The High Availability Linux Project
Linux-HA provides sophisticated high-availability (failover) capabilities on a wide range of platforms, supporting several tens of thousands of mission critical sites.
=> Project web site
Ultra Monkey is a project to create load balanced and highly available network services. For example a cluster of web servers that appear as a single web server to end-users. The service may be for end-users across the world connected via the internet, or for enterprise users connected via an intranet.
Ultra Monkey makes use of the Linux operating system to provide a flexible solution that can be tailored to a wide range of needs. From small clusters of only two nodes to large systems serving thousands of connections per second.
=> Project web site
Personally, I’ve worked with both LVS and Red Hat Cluster Suite and I highly recommend these softwares.
Xen is one of the leading Virtualization software. You can use Xen virtualization to implement HA clusters. However, there are few issues you must be aware of while handling failures in a high-availability environment. This article explains configuration options using Xen:
The idea of using virtual machines to build high available clusters is not new. Some software companies claim that virtualization is the answer to your HA problems, off course that’s not true. Yes, you can reduce downtime by migrating virtual machines to another physical machine for maintenance purposes or when you think hardware is about to fail, but if an application crashes you still need to make sure another application instance takes over the service. And by the time your hardware fails, it’s usually already too late to initiate the migration.
So, for each and every application you still need to look at whether you want to have it constantly available, if you can afford the application to be down for some time, or if your users won’t mind having to relogin when one server fails.
=> Using Xen for High Availability Clusters [onlamp.com]
Intel has just released source code for Fibre Channel over Ethernet (FCoE). It provides some Fibre Channel protocol processing as well as the encapsulation of FC frames within Ethernet packets. FCoE will allow systems with an Ethernet adapter and a Fibre Channel Forwarder to login to a Fibre Channel fabric (the FCF is a “gateway” that bridges the LAN and the SAN). That fabric login was previously reserved exclusively for Fibre Channel HBAs. This technology reduces complexity in the data center by aiding network convergence. It is targeted for 10Gps Ethernet NICs but will work on any Ethernet NIC supporting pause frames. Intel will provide a Fibre Channel protocol processing module as well as an Ethernet based transport module. The Open-FC module acts as a LLD for SCSI and the Open-FCoE transport uses net_device to send and receive packets.
This is good news. I think one can compare bandwidth and throughput for copper and fiber Ethernet. If you are going to use copper you need to stay within 15m of the switch. This solution will try to bring down cost. One can connect to 8-10 server to central database server with 10G and there could be few more applications.
=> Open FCoE project home page
My previous article related to iSCSI storage and NAS storage brought a couple of questions. An interesting question from my mail bag:
I’ve 5 Debian Linux servers with HP SAN box. Should I boot from SAN?
No, use centralized network storage for shared data or high availability configuration only. Technically you can boot and configure system. However I don’t recommend booting from SAN or any other central server until and unless you need diskless nodes:
[a] Use local storage – Always use local storage for /boot and / (root) filesystem
[b] Keep it simply – Booting from SAN volumes is complicated procedure. Most operating systems are not designed for this kind of configuration. You need to modify scripts and booting procedure.
[c] SAN booting support – Your SAN vendor must support platform booting a Linux server. You need to configure HBA and SAN according to vendor specification. You must totally depend upon SAN vendor for drivers and firmware (HBA Bios) to get thing work properly. General principle – don’t put all your eggs in one basket err one vendor 😉
[d] Other factors – Proper fiber channel topology must be used. Make sure Multipathing and redundant SAN links are used. The boot disk LUN is dedicated to a single host. etc
As you can see, complications started to increases, hence I don’t recommend booting from SAN.
If you have 4 GB or more RAM use the Linux kernel compiled for PAE capable machines. Your machine may not show up total 4GB ram. All you have to do is install PAE kernel package.
Explains how to create filesystems greater than 2 Terabytes under any Linux distribution. How can I create filesystems greater than 2 Terabytes?
Yesterday I wrote about increasing local port range with net.ipv4.ip_local_port_range proc file. There is also /proc/sys/kernel/pid_max file, which specifies the value at which PIDs wrap around (i.e., the value in this file is one greater than the maximum PID). The default value for this file, 32768, results in the same range of PIDs as on earlier kernels (<=2.4). The value in this file can be set to any value up to 2^22 (PID_MAX_LIMIT, approximately 4 million).
If your Linux server is opening lots of outgoing network connections, you need to increase local port range. By default range is small. For example a squid proxy server can come under fire if it runs out of ports. Other example includes heavy traffic network servers, like nginx load balancers, LXD vm and more.