Linux bond or team multiple network interfaces (NIC) into single interface

Posted by Vivek Gite [Last updated: November 4, 2006]

Finally today I had implemented NIC bounding (bind both NIC so that it works as a single device). We have two Dell servers that need setup with Intel Dual Gig NIC. My idea is to improve performance by pumping out more data from both NIC without using any other method.

This box act as heavy duty ftp server. Each night I need to transfer over 200GB data from this box to another box. Therefore, the network would be setup is two servers on a switch using dual network cards. I am using Red Hat enterprise Linux version 4.0.

Linux allows binding multiple network interfaces into a single channel/NIC using special kernel module called bonding. According to official bonding documentation, "The Linux bonding driver provides a method for aggregating multiple network interfaces into a single logical "bonded" interface. The behavior of the bonded interfaces depends upon the mode; generally speaking, modes provide either hot standby or load balancing services. Additionally, link integrity monitoring may be performed."

Setting up bounding is easy with RHEL v4.0.

Step #1: Create a bond0 configuration file

Red Hat Linux stores network configuration in /etc/sysconfig/network-scripts/ directory. First, you need to create bond0 config file:
# vi /etc/sysconfig/network-scripts/ifcfg-bond0Append following lines to it:DEVICE=bond0 IPADDR=192.168.1.20 NETWORK=192.168.1.0 NETMASK=255.255.255.0 USERCTL=no BOOTPROTO=none ONBOOT=yesReplace above IP address with your actual IP address. Save file and exit to shell prompt.

Step #2: Modify eth0 and eth1 config files:

Open both configuration using vi text editor and make sure file read as follows for eth0 interface# vi /etc/sysconfig/network-scripts/ifcfg-eth0Modify/append directive as follows:DEVICE=eth0 USERCTL=no ONBOOT=yes MASTER=bond0 SLAVE=yes BOOTPROTO=noneOpen eth1 configuration file using vi text editor:# vi /etc/sysconfig/network-scripts/ifcfg-eth1Make sure file read as follows for eth1 interface:DEVICE=eth1 USERCTL=no ONBOOT=yes MASTER=bond0 SLAVE=yes BOOTPROTO=noneSave file and exit to shell prompt.

Step # 3: Load bond driver/module

Make sure bonding module is loaded when the channel-bonding interface (bond0) is brought up. You need to modify kernel modules configuration file:# vi /etc/modprobe.confAppend following two lines:alias bond0 bonding options bond0 mode=balance-alb miimon=100Save file and exit to shell prompt. You can learn more about all bounding options in kernel source documentation file (click here to read file online).

Step # 4: Test configuration

First, load the bonding module:# modprobe bondingRestart networking service in order to bring up bond0 interface:# service network restartVerify everything is working:# less /proc/net/bonding/bond0Output:

Bonding Mode: load balancing (round-robin)
MII Status: up
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth0
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:0c:29:c6:be:59

Slave Interface: eth1
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:0c:29:c6:be:63

List all interfaces:# ifconfigOutput:

bond0     Link encap:Ethernet  HWaddr 00:0C:29:C6:BE:59
 inet addr:192.168.1.20  Bcast:192.168.1.255  Mask:255.255.255.0
 inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link
 UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
 RX packets:2804 errors:0 dropped:0 overruns:0 frame:0
 TX packets:1879 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:0
 RX bytes:250825 (244.9 KiB)  TX bytes:244683 (238.9 KiB)

eth0      Link encap:Ethernet  HWaddr 00:0C:29:C6:BE:59
 inet addr:192.168.1.20  Bcast:192.168.1.255  Mask:255.255.255.0
 inet6 addr: fe80::20c:29ff:fec6:be59/64 Scope:Link
 UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
 RX packets:2809 errors:0 dropped:0 overruns:0 frame:0
 TX packets:1390 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:1000
 RX bytes:251161 (245.2 KiB)  TX bytes:180289 (176.0 KiB)
 Interrupt:11 Base address:0x1400

eth1      Link encap:Ethernet  HWaddr 00:0C:29:C6:BE:59
 inet addr:192.168.1.20  Bcast:192.168.1.255  Mask:255.255.255.0
 inet6 addr: fe80::20c:29ff:fec6:be59/64 Scope:Link
 UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
 RX packets:4 errors:0 dropped:0 overruns:0 frame:0
 TX packets:502 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:1000
 RX bytes:258 (258.0 b)  TX bytes:66516 (64.9 KiB)
 Interrupt:10 Base address:0x1480

Now you have bond multiple network interfaces into a single channel (NIC). Read the official howto it covers following additional topics:

VLAN Configuration
Cisco switch related configuration
Advanced routing and troubleshooting

Tell us how we're doing: Please answer a few questions about your experience to help us improve nixCraft.

Discussion on This Article:

Anonymous Says: June 6th, 2006 at 11:32 pm
Many thanks for quick instructions. It worked like a charm.

Just one more hint:
Enable spanning tree for switches. I had noticed the network speed reduction when spanning tree was not enabled.

Just my two 2 cents
Anonymous Says: September 5th, 2006 at 7:47 pm
Thanks for the quick help!! bonding r0x!! Just make sure that eth0 and eth1 are up when you put up the bond0. And be careful if you’re doing this over a remote ssh. *thumbs up*
mattcosa Says: November 30th, 2006 at 1:42 pm
Did you notice that your /proc/net/bonding/bond0 was reporting incorrect settings based on your mode and other settings?

I’m having the same problem.
zen Says: January 30th, 2007 at 2:54 pm
hmm. did anyone experience what i did?

when i do a ifdown eth0, the ping timed out. it doesn’t seem to be bonding well.
Gaurav Says: February 14th, 2007 at 6:11 am
What will be the behaviour if eth0 and eth1 interfaces does not have IP addresses and they share the IP address of bond0 in a failover scenario.
Will there be any entey for eth0 and eth1 in /proc/net/route????

- gaurav
H8injected Says: March 15th, 2007 at 12:48 pm
bond0 config file needed another line in my configuration:
GATEWAY=192.168.0.1
seno Says: April 15th, 2007 at 6:57 pm
Can you bond more than 2 nics? e.g 4 NICs?
Seno Says: April 15th, 2007 at 7:03 pm
Reason why I ask is because I want a big fat trunk of say 4×1Gig Nic’s and then run many virtulized O/S that runs it’s connection virtually though bond0.
nixcraft Says: April 15th, 2007 at 7:22 pm
Seno,

How many bonding devices can I have?

There is no limit.

How many slaves can a bonding device have? OR Can you bond more than 2 nics?
This is limited only by the number of network interfaces (NIC) Linux supports and/or the number of network cards you can place in your system.

HTH
Please Says: May 10th, 2007 at 5:15 am
HI

i have 2 uplink and 1 internal link.i want to use 2 uplink at the same time.i installed core 5.
so how can i do?
my information
etho
ip-10.241.215.163
nesk-255.255.255.240
GW-10.241.215.161

eth2
ip-10.255.8.104
nesk-255.255.255.252.0
GW-10.255.8.1

eth1
ip-192.168.1.1
nesk-255.255.255.0

Thanks
Please
Andres Says: June 4th, 2007 at 4:44 pm
Thanks!
It works very well…
Just add this :
GATEWAY=192.168.1.1 (replace with your gateway)

To the bond config file

thanks!
Rizwan Says: August 29th, 2007 at 10:53 am
I have 4 DSL modems connected with 4 different ISP’s my scenario is
a) My FC-2 machine with LAN IP=192.168.10.1 and Bond0 IP=192.168.1.1
b) Modem-A LAN IP= 192.168.1.2 , ext IP=xxx.xxx.xxx.xxx
c) Modem-B LAN IP=192.168.1.3, ext IP=xxx.xxx.xxx.xxx
d) Modem-C LAN IP=192.168.1.4, ext IP=xxx.xxx.xxx.xxx
e) Modem-D LAN IP=192.168.1.5, ext IP=xxx.xxx.xxx.xxx

Modem-A, B, C, and D LAN connected with my FC-2 machine, and all 4 interfaces of my machine are in Bond0,
Now please help me what default Gateway I should set in my FC-2 machine?>??? or I have to set 4 gateways in my machine??
and will this configuration works??
my scenario will also listed in my home page, please visit Linux section for better understanding.

Thanks in advance.
Rizwan.
Karachi, Pakistan.
homepage: http://freewaresolutions.150m.com
David Kittell Says: September 21st, 2007 at 8:39 am
Great tutorial however on FC7 and likely other distributions you have to manually assign an IP to the bond0. Not at all a big deal though

Great work.
David Kittell Says: September 21st, 2007 at 8:43 am
What I mean by that is that after reboot on FC7 it appears you have to reassign the bond0 causing you to do the “vi /etc/sysconfig/network-scripts/ifcfg-bond0″ part again and yes I did :wq!

May just be an issue with my build.

Again great work
shyam Says: December 4th, 2007 at 2:08 pm
I am using this configuration on centos 5.0 and when i do service network restart i am getting warning hardware address conflicts with eth0 change hardware address.

Please suggest me on the same.

Thanks in advance
Shyam
nisha Says: December 19th, 2007 at 2:16 am
Perfect guide! My NIC bond went without a single hitch! Thanks and keep the guides coming…
Kyle Says: February 5th, 2008 at 6:20 pm
I did this and it worked on my FC4 machine, but it doesn’t show eth1 in the ifconfig. I am not sure if it is bonded or not. Can you help me with this problem?
naim Says: February 26th, 2008 at 9:25 am
I did this with FC8, I’m newbie and i am confused that how can i bond two dsl modem together is this the way or this is some thing else? or this is only bonding two nic for heavy traffic/load balancing in LAN?
p Says: March 31st, 2008 at 9:48 am
This is only for bonding two nics onto the same network, to provide resilience or extra throughput. It’s no use if you want to balance across multiple networks.
krishna Says: April 27th, 2008 at 9:25 am
how to enable MII Status for eth0. Although i have done all the above steps, but its creating problem.

/proc/net/bonding/bond0, snippet :

Slave Interface: eth0
MII Status: down
Link Failure Count: 0
Permanent HW addr: 00:1b:a0:26:cb:78
E71 Says: May 23rd, 2008 at 4:39 pm
Does anyone know if /proc/net/bonding/bond0 will display ‘MII status: up’ ONLY when your NICs fully support Link Aggregation?

It’s showing as UP for everything and ifconfig produces something similar to what’s shown in this article except the NICs (eth2,eth3) don’t show an inet value… Is that normal?
Andy Says: July 24th, 2008 at 8:05 am
Great tut but will bonding work with dynamic IP’s as I do not have static IP’s from my ISP
yandy Says: August 6th, 2008 at 3:38 am
Great guide…..managed to configure almost everything but after reboot of server, only managed to see bond0 in up status. I need to execute the network script (service network start) again in order to have eth0 in up status. Kindly advise how can this be resolve??? Thank you.
Mandar Says: August 13th, 2008 (4 weeks ago) at 3:29 pm
Very nice explanation!
Mike Baysek Says: September 2nd, 2008 (4 days ago) at 4:25 pm
If you are considering teaming NICs, remember the limitations of the bus itself.

For example, if you think putting 4 Gigabit cards in your PCI bus will get you more speed, you are sadly mistaken! PCI bus is limited to maximum of around 133 MB/s. Putting two gigabit NICs on PCI will be a waste of time. Better to have at least PCI-X, or even better, PCI-e before you think about teaming your adapters.

Remember too, that even on-board NICs usually share the PCI bus with the PCI slots. So adding a gig card to a machine with an onboard gig adapter would also usually be a waste of time unless your bus was, again, PCI-X or better.