≡ Menu

high speed

FAQ Updates: Dec/22/2011

Our FAQ section is updated in last few days with new howto:

Make the most of nixCraft FAQ Section

Get intimated about our new howtos / faqs as soon as it is released via RSS feed OR follow us on twitter OR join our 16k+ facebook fans for more tips and news about Linux.

Parallel supercomputers and computer clusters needs to use failover and Infiniband multipathing to provide non stop computing. InfiniBand is a switched fabric communications link primarily used in high-performance computing. Its features include quality of service and failover, and it is designed to be scalable. The InfiniBand architecture specification defines a connection between processor nodes and high performance I/O nodes such as storage devices. It is a superset of the Virtual Interface Architecture.

InfiniBand theoretical throughput (speed)

InfiniBand provides high speed data transfer. For example USB 2.0 provides 480Mb/s data transfer or Gigagbit Ethernet support 1,000Mb/s, while 12X InfiniBand provides 96 Gbit/s in quad configuration. It also supports both copper and optical cabling. Following table shows effective theoretical throughput in different configurations:

InfiniBandSingleDoubleQuad
1X2 Gbit/s4 Gbit/s8 Gbit/s
4X 8 Gbit/s16 Gbit/s32 Gbit/s
12X24 Gbit/s48 Gbit/s96 Gbit/s

The InfiniBand Project

The InfiniBand Architecture (IBA) is an industry standard that defines a new high-speed switched fabric subsystem designed to connect processor nodes and I/O nodes to form a system area network. This new interconnect method moves away from the local transaction-based I/O model across busses to a remote message-passing model across channels. The architecture is independent of the host operating system (OS) and the processor platform.

IBA provides both reliable and unreliable transport mechanisms in which messages are enqueued for delivery between end systems. Hardware transport protocols are defined that support reliable and unreliable messaging (send/receive), and memory manipulation semantics (e.g., RDMA read/write) without software intervention in the data transfer path.

Linux and infiniband support

Most enterprise Linux distribution (such as RHEL 4.5 / 5, CentOS / Novell Linux) has support for Infiniband (IPoIB), multipathing and failover. Linux kernel v2.6.11 and above has support for IPoIB and related technologies. The OpenFabrics Alliance is creating an open source software stack for InfiniBand and iWARP that includes the "IBVerbs" library.

I may get chance to play with infiniband based devices and Linux in near future :D

List of open source cluster management systems

M. Shuaib Khan has published a list of open-source cluster management systems.

Personally, I had used openMosix and Red Hat Cluster software (which is also based upon open source software funded by Red Hat).

From the article: In computing world, the term "cluster" refers to a group of independent computers combined through software and networking, which is often used to run highly compute-intensive jobs. With a cluster, you can build a high-speed supercomputer out of hundreds or even thousands of relatively low-speed systems. Cluster management software offers an easy-to-use interface for managing clusters, and automates the process of queuing jobs, matching the requirements of a job and the resources available to the cluster, and migrating jobs across the cluster:

=> openMosix
=> Kerrighed
=> OpenSSI
=> Gluster
=> BOINC

Read this article it offers feature, cons and pros of each solution.