RHEL: Linux Bond / Team Multiple Network Interfaces (NIC) Into a Single Interface

by on April 3, 2006 · 131 comments· LAST UPDATED September 4, 2011

in , ,

Finally, today I had implemented NIC bounding (bind both NIC so that it works as a single device). Bonding is nothing but Linux kernel feature that allows to aggregate multiple like interfaces (such as eth0, eth1) into a single virtual link such as bond0. The idea is pretty simple get higher data rates and as well as link failover. The following instructions were tested on:

  1. RHEL v4 / 5 / 6 amd64
  2. CentOS v5 / 6 amd64
  3. Fedora Linux 13 amd64 and up.
  4. 2 x PCI-e Gigabit Ethernet NICs with Jumbo Frames (MTU 9000)
  5. Hardware RAID-10 w/ SAS 15k enterprise grade hard disks.
  6. Gigabit switch with Jumbo Frame

This server act as an heavy duty ftp, and nfs file server. Each, night a perl script will transfer lots of data from this box to a backup server. Therefore, the network would be setup on a switch using dual network cards. I am using Red Hat enterprise Linux version 4.0. But, the inductions should work on RHEL 5 and 6 too.

Say Hello To bounding Driver

Linux allows binding of multiple network interfaces into a single channel/NIC using special kernel module called bonding. According to official bonding documentation:

The Linux bonding driver provides a method for aggregating multiple network interfaces into a single logical "bonded" interface. The behavior of the bonded interfaces depends upon the mode; generally speaking, modes provide either hot standby or load balancing services. Additionally, link integrity monitoring may be performed.

Step #1: Create a Bond0 Configuration File

Red Hat Enterprise Linux (and its clone such as CentOS) stores network configuration in /etc/sysconfig/network-scripts/ directory. First, you need to create a bond0 config file as follows:
# vi /etc/sysconfig/network-scripts/ifcfg-bond0
Append the following linest:

 
DEVICE=bond0
IPADDR=192.168.1.20
NETWORK=192.168.1.0
NETMASK=255.255.255.0
USERCTL=no
BOOTPROTO=none
ONBOOT=yes
 

You need to replace IP address with your actual setup. Save and close the file.

Step #2: Modify eth0 and eth1 config files

Open both configuration using a text editor such as vi/vim, and make sure file read as follows for eth0 interface
# vi /etc/sysconfig/network-scripts/ifcfg-eth0
Modify/append directive as follows:
DEVICE=eth0
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none

Open eth1 configuration file using vi text editor, enter:
# vi /etc/sysconfig/network-scripts/ifcfg-eth1
Make sure file read as follows for eth1 interface:
DEVICE=eth1
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none

Save and close the file.

Step # 3: Load bond driver/module

Make sure bonding module is loaded when the channel-bonding interface (bond0) is brought up. You need to modify kernel modules configuration file:# vi /etc/modprobe.conf
Append following two lines:alias bond0 bonding
options bond0 mode=balance-alb miimon=100

Save file and exit to shell prompt. You can learn more about all bounding options by clicking here).

Step # 4: Test configuration

First, load the bonding module, enter:
# modprobe bonding
Restart the networking service in order to bring up bond0 interface, enter:
# service network restart
Make sure everything is working. Type the following cat command to query the current status of Linux kernel bounding driver, enter:
# cat /proc/net/bonding/bond0
Sample outputs:

Bonding Mode: load balancing (round-robin)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 200
Down Delay (ms): 200
Slave Interface: eth0
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:0c:29:c6:be:59
Slave Interface: eth1
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:0c:29:c6:be:63

To kist all network interfaces, enter:
# ifconfig
Sample outputs:

bond0     Link encap:Ethernet  HWaddr 00:0C:29:C6:BE:59
 inet addr:192.168.1.20  Bcast:192.168.1.255  Mask:255.255.255.0
 inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link
 UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
 RX packets:2804 errors:0 dropped:0 overruns:0 frame:0
 TX packets:1879 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:0
 RX bytes:250825 (244.9 KiB)  TX bytes:244683 (238.9 KiB)
eth0      Link encap:Ethernet  HWaddr 00:0C:29:C6:BE:59
 inet addr:192.168.1.20  Bcast:192.168.1.255  Mask:255.255.255.0
 inet6 addr: fe80::20c:29ff:fec6:be59/64 Scope:Link
 UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
 RX packets:2809 errors:0 dropped:0 overruns:0 frame:0
 TX packets:1390 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:1000
 RX bytes:251161 (245.2 KiB)  TX bytes:180289 (176.0 KiB)
 Interrupt:11 Base address:0x1400
eth1      Link encap:Ethernet  HWaddr 00:0C:29:C6:BE:59
 inet addr:192.168.1.20  Bcast:192.168.1.255  Mask:255.255.255.0
 inet6 addr: fe80::20c:29ff:fec6:be59/64 Scope:Link
 UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
 RX packets:4 errors:0 dropped:0 overruns:0 frame:0
 TX packets:502 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:1000
 RX bytes:258 (258.0 b)  TX bytes:66516 (64.9 KiB)
 Interrupt:10 Base address:0x1480

Read the official bounding howto which covers the following additional topics:

  • VLAN Configuration
  • Cisco switch related configuration
  • Advanced routing and troubleshooting
TwitterFacebookGoogle+PDF versionFound an error/typo on this page? Help us!
This entry is 1 of 2 in the Linux NIC Interface Bonding (aggregate multiple links) Tutorial series. Keep reading the rest of the series:
  1. Red Hat (RHEL/CentOS) Linux Bond or Team Multiple Network Interfaces (NIC) into a Single Interface
  2. Debian / Ubuntu Linux Configure Bonding [ Teaming / Aggregating NIC ]

{ 131 comments… read them below or add one }

1 Anonymous June 6, 2006 at 11:32 pm

Many thanks for quick instructions. It worked like a charm.

Just one more hint:
Enable spanning tree for switches. I had noticed the network speed reduction when spanning tree was not enabled.

Just my two 2 cents

Reply

2 Avid June 13, 2013 at 6:47 pm

Could you please tell me how to enable spanning tree?

Reply

3 mansour ghanbarig November 16, 2014 at 6:22 am

hi
best guide for bonding
tnx

Reply

4 Anonymous September 5, 2006 at 7:47 pm

Thanks for the quick help!! bonding r0x!! Just make sure that eth0 and eth1 are up when you put up the bond0. And be careful if you’re doing this over a remote ssh. *thumbs up*

Reply

5 mattcosa November 30, 2006 at 1:42 pm

Did you notice that your /proc/net/bonding/bond0 was reporting incorrect settings based on your mode and other settings?

I’m having the same problem.

Reply

6 zen January 30, 2007 at 2:54 pm

hmm. did anyone experience what i did?

when i do a ifdown eth0, the ping timed out. it doesn’t seem to be bonding well.

Reply

7 Clement January 24, 2013 at 11:03 am

same problem here, when i ifconfig down for eth1 there is no drop in network but when i do the same ifconfig down for eth0 i have a drop in network connectivity, please help..!!

Reply

8 Sunil March 30, 2014 at 10:45 am

myself getting the same problem, if someone has a fix pls help !!!

Reply

9 Gaurav February 14, 2007 at 6:11 am

What will be the behaviour if eth0 and eth1 interfaces does not have IP addresses and they share the IP address of bond0 in a failover scenario.
Will there be any entey for eth0 and eth1 in /proc/net/route????

– gaurav

Reply

10 Mike Smith April 11, 2014 at 9:44 am

When eth0 and eth1 are bonded, there are entries for them still in /proc/net/route, but bond0 is listed first so gets the traffic.

Reply

11 H8injected March 15, 2007 at 12:48 pm

bond0 config file needed another line in my configuration:
GATEWAY=192.168.0.1

Reply

12 seno April 15, 2007 at 6:57 pm

Can you bond more than 2 nics? e.g 4 NICs?

Reply

13 Seno April 15, 2007 at 7:03 pm

Reason why I ask is because I want a big fat trunk of say 4x1Gig Nic’s and then run many virtulized O/S that runs it’s connection virtually though bond0.

Reply

14 nixCraft April 15, 2007 at 7:22 pm

Seno,

How many bonding devices can I have?

There is no limit.

How many slaves can a bonding device have? OR Can you bond more than 2 nics?
This is limited only by the number of network interfaces (NIC) Linux supports and/or the number of network cards you can place in your system.

HTH

Reply

15 Please May 10, 2007 at 5:15 am

HI

i have 2 uplink and 1 internal link.i want to use 2 uplink at the same time.i installed core 5.
so how can i do?
my information
etho
ip-10.241.215.163
nesk-255.255.255.240
GW-10.241.215.161

eth2
ip-10.255.8.104
nesk-255.255.255.252.0
GW-10.255.8.1

eth1
ip-192.168.1.1
nesk-255.255.255.0

Thanks
Please

Reply

16 Andres June 4, 2007 at 4:44 pm

Thanks!
It works very well…
Just add this :
GATEWAY=192.168.1.1 (replace with your gateway)

To the bond config file

thanks!

Reply

17 Rizwan August 29, 2007 at 10:53 am

I have 4 DSL modems connected with 4 different ISP’s my scenario is
a) My FC-2 machine with LAN IP=192.168.10.1 and Bond0 IP=192.168.1.1
b) Modem-A LAN IP= 192.168.1.2 , ext IP=xxx.xxx.xxx.xxx
c) Modem-B LAN IP=192.168.1.3, ext IP=xxx.xxx.xxx.xxx
d) Modem-C LAN IP=192.168.1.4, ext IP=xxx.xxx.xxx.xxx
e) Modem-D LAN IP=192.168.1.5, ext IP=xxx.xxx.xxx.xxx

Modem-A, B, C, and D LAN connected with my FC-2 machine, and all 4 interfaces of my machine are in Bond0,
Now please help me what default Gateway I should set in my FC-2 machine?>??? or I have to set 4 gateways in my machine??
and will this configuration works??
my scenario will also listed in my home page, please visit Linux section for better understanding.

Thanks in advance.
Rizwan.
Karachi, Pakistan.
homepage: http://freewaresolutions.150m.com

Reply

18 David Kittell September 21, 2007 at 8:39 am

Great tutorial however on FC7 and likely other distributions you have to manually assign an IP to the bond0. Not at all a big deal though

Great work.

Reply

19 David Kittell September 21, 2007 at 8:43 am

What I mean by that is that after reboot on FC7 it appears you have to reassign the bond0 causing you to do the “vi /etc/sysconfig/network-scripts/ifcfg-bond0″ part again and yes I did :wq! :-)

May just be an issue with my build.

Again great work

Reply

20 shyam December 4, 2007 at 2:08 pm

I am using this configuration on centos 5.0 and when i do service network restart i am getting warning hardware address conflicts with eth0 change hardware address.

Please suggest me on the same.

Thanks in advance
Shyam

Reply

21 nisha December 19, 2007 at 2:16 am

Perfect guide! My NIC bond went without a single hitch! Thanks and keep the guides coming… ;-)

Reply

22 Kyle February 5, 2008 at 6:20 pm

I did this and it worked on my FC4 machine, but it doesn’t show eth1 in the ifconfig. I am not sure if it is bonded or not. Can you help me with this problem?

Reply

23 naim February 26, 2008 at 9:25 am

I did this with FC8, I’m newbie and i am confused that how can i bond two dsl modem together is this the way or this is some thing else? or this is only bonding two nic for heavy traffic/load balancing in LAN?

Reply

24 p March 31, 2008 at 9:48 am

This is only for bonding two nics onto the same network, to provide resilience or extra throughput. It’s no use if you want to balance across multiple networks.

Reply

25 krishna April 27, 2008 at 9:25 am

how to enable MII Status for eth0. Although i have done all the above steps, but its creating problem.

/proc/net/bonding/bond0, snippet :

Slave Interface: eth0
MII Status: down
Link Failure Count: 0
Permanent HW addr: 00:1b:a0:26:cb:78

Reply

26 E71 May 23, 2008 at 4:39 pm

Does anyone know if /proc/net/bonding/bond0 will display ‘MII status: up’ ONLY when your NICs fully support Link Aggregation?

It’s showing as UP for everything and ifconfig produces something similar to what’s shown in this article except the NICs (eth2,eth3) don’t show an inet value… Is that normal?

Reply

27 Andy July 24, 2008 at 8:05 am

Great tut but will bonding work with dynamic IP’s as I do not have static IP’s from my ISP

Reply

28 yandy August 6, 2008 at 3:38 am

Great guide…..managed to configure almost everything but after reboot of server, only managed to see bond0 in up status. I need to execute the network script (service network start) again in order to have eth0 in up status. Kindly advise how can this be resolve??? Thank you.

Reply

29 Mandar August 13, 2008 at 3:29 pm

Very nice explanation!

Reply

30 Mike Baysek September 2, 2008 at 4:25 pm

If you are considering teaming NICs, remember the limitations of the bus itself.

For example, if you think putting 4 Gigabit cards in your PCI bus will get you more speed, you are sadly mistaken! PCI bus is limited to maximum of around 133 MB/s. Putting two gigabit NICs on PCI will be a waste of time. Better to have at least PCI-X, or even better, PCI-e before you think about teaming your adapters.

Remember too, that even on-board NICs usually share the PCI bus with the PCI slots. So adding a gig card to a machine with an onboard gig adapter would also usually be a waste of time unless your bus was, again, PCI-X or better.

Reply

31 Hassan Tariq September 24, 2008 at 6:34 am

I exactly need the same kind of solution in Windows 2003 Server.Please help me out on this!

Reply

32 mike October 7, 2008 at 3:29 pm

Does anyone know how to trunc two bonding Interfaces to a single one with active-passive mode?

Reply

33 tecnimxv October 21, 2008 at 8:17 pm

I would like to ask if it is possible to bond ethernet connection with a wireless one?

Reply

34 gael November 6, 2008 at 3:42 pm

My OS: xen-dom0 on centos5

I am setting up bonding at the moment and I was wondering why we need:

ONBOOT=yes

I tried with “ONBOOT=no” and my bond interface is brought up at startup with both slaves…

As far as I understand bonding, it seems that when the bonding interface is starting up, it brings up the required slave interfaces.

Please correct me if I am wrong :-) And if you can explain why it would be excellent!

Cheers
Gael

Reply

35 Tri Aji Wibowo October 5, 2011 at 10:24 am

Hi Gael,

I think you must be hash your config MASTER=bond0.
Try this:

DEVICE=eth(x)
USERCTL=no
ONBOOT=yes
#MASTER=bond(x)
SLAVE=yes
BOOTPROTO=none

-TA-

Reply

36 USMAN December 4, 2008 at 7:17 am

i m new with linux.i m using redhat linux 9.0 with squid proxy server for my netcafe.i have two broadband conections.the setting of external interfaces(broadeband) are following
1)
192.168.0.1 ip
255.255.255.0 mask
192.168.0.1 gateay
2)
192.168.1.1 ip
255.255.255.0 mask
192.168.1.1 gateway

my internal nic ip is 10.0.0.1 which is for out put.plz send me the bond setting with these ips.i will be very thankfull to you.

Reply

37 AB June 19, 2010 at 7:59 am

Hi There, I have the same situation, did you get bonding working for your network. Please share the information, would be great. Thanks AB

Reply

38 vijay January 29, 2009 at 1:43 pm

i want to install inbuilt squid proxy in fedora server 9 pls guide me step by step configuration

i am using tata broadband

with regards
vijay

Reply

39 LinuxStudent February 10, 2009 at 5:15 pm

Hey Man,

Solid info provided with minimum efforts…..

Reply

40 nima February 21, 2009 at 6:44 am

Hi
I have 2 links (5Mb and 2Mb) from 2 ISP, Does this solution give traffic load balancing possibility as if I have 7Mb bandwidth ?
thanks for your any help

Reply

41 Linux-student February 24, 2009 at 6:23 am

i m using linux fedora 9.0 with squid proxy server for my internet cafe. I have two broadband connections. The setting of external interfaces(broadband) are following

1) Bandwidth – 2 Mb uplink
10.254.55.16 ip
255.255.252.0 mask
10.254.55.1 gateay

2)Bandwidth – 3 Mb uplink
10.254.22.45 ip
255.255.252.0 mask
192.168.22.1 gateway

3)
LAN IP
192.168.1.2
255.255.255.0
192.168.1.1

What I need to get traffic load balancing. Could you Please advice me.

thank in advance
KA

Reply

42 TJ March 1, 2009 at 4:32 pm

Hi,

How do I connect the slave interface eth0, eth1 to the LAN switch, both interfaces need to connect or just only one ? …

If both eth0 and eth1 are connected, does the LAN switch select one higher priority than other ?

Thanks
TJ

Reply

43 Elena March 3, 2009 at 6:41 am

Hi

I have a bond created. But if I bring both interfaces down, and then back up, they are not making link up; they appear as down.. They will only come up if I bring up the whole bond. Where can be the problem?

Reply

44 kawaii avon April 3, 2009 at 7:23 am

how come that u set the mode in balance-alb yet when you type ifconfig, the configuration shows round robin mode..?

Reply

45 shay May 6, 2009 at 1:18 pm

options bond0 mode=balance-alb miimon=100
should be
options bonding mode=balance-alb miimon=100

Reply

46 Dima May 12, 2009 at 7:28 am

Thanks for great article! ;)

Reply

47 Eldo May 14, 2009 at 7:29 am

Hi,

I try to configure bond interface in VMWARE with 4 ethernet cards, I follow the steps mentioned above. The problem is only bond interface have ip, there is no ip for eth0,eth1,eth2,eth3. I tried this in RHEL 5.1

Reply

48 Mike Roberts May 20, 2009 at 7:22 pm

I am currently having issues with bonding on a RHEL 5 machine. I have the bonding setup correctly and when I run the ifconfig command it looks to be up, however when I attempt to ping this machine from another computer I get no response. However, if I start a ping from the RHEL 5 comptuer and then attempt to ping from another computer it responds. Any insight would be greatly appreciated.

FYI, I am using “active-backup” mode.

Reply

49 James J August 31, 2010 at 7:16 pm

Did you ever find the fix for this? I’m having the same issue.

Reply

50 Jarvis Semien June 3, 2011 at 5:21 pm

Please let me know if you guys found a fix for this. Thanks in advance

Reply

51 Mike Smith April 11, 2014 at 10:04 am

This is probably because the bonded ethernet cards appear to change MAC address and the MAC address moves from one ethernet interface to another. If you are using switches, these need to be trained so that the switch knows where the MAC has moved to – that’s why you need the ping from your RHEL 5 machine. In current implementations of the bonding driver, it is possible to configure the bonding driver to send out ARP packets on the active adapter at switchover (the num_grat_arp parameter) and this ARP will help train any external switches. Make sure you read https://www.kernel.org/doc/Documentation/networking/bonding.txt.

Reply

52 loptos June 24, 2009 at 2:00 pm

Is there any command, which would swith the active interface in the bond?
Example: bond0 consists of eth0 and eth1
now eth1 is active
enter command
now eth0 is active
Also I’m searching for command, which would check, which interface in bond is active.
(Suse linux)
Thanks, Peter

Reply

53 simon August 5, 2009 at 3:50 pm

Been searching for the answer to loptos above and thought I’d leave it here.

To find which interface is active have a look in /proc/net/bonding/bond0 (mentioned earlier).

To change which is active have a look at the ifenslave command and try something like this:
ifenslave -c bond0 eth1

Reply

54 fabrice August 21, 2009 at 7:36 am

In my case I wasn’t able to start a second bonding interface on a RHEL4.
bond0 : eth2 and eth3
bond1 : eth4 and eth5

Error message was :
“Bringing up interface bond1: bonding device bond1 does not seem to be present, delaying initialization.”

after 2 days of google and test i found a working configuration. My problem was in the modprobe.conf configuration file.

Here are the bonding configuration in the modprobe.conf working for me :

# allow 2 bounding interfaces
options bonding max_bonds=2

# bond0
install bond0 /sbin/modprobe -a eth2 eth3 && /sbin/modprobe bonding
alias bond0 bonding
options bond0 mode=x,miimon=100,primary=eth2,etc.

# bond1
install bond1 /sbin/modprobe -a eth4 eth5 && /sbin/modprobe bonding
alias bond1 bonding
options bond1 mode=x,miimon=100,primary=eth4,etc.

Hop it will help someone.

Fabrice

Reply

55 EllisGL June 22, 2011 at 2:39 am

Thanks. That did the trick. I had issues with it half working and this totally fixed my issues.

Reply

56 Shreejyot March 13, 2012 at 5:12 pm

This really works. You are awesome…!!!

Reply

57 Deiveegan August 27, 2009 at 4:11 am

Hello sir,
here we are using 3 squid (proxy )in Redhat 4 and sharing internet for 1) for wifi 2) for Lan and 3) for staff … if one squid dead that people are suffering ..if there is any load balancing server for squid ..how share there load if one squid dead…

Thanking you
Deiveegan.S
SASTRA University
Thanjavur-Tamilnadu-India

Reply

58 satya September 28, 2009 at 11:53 pm

Thanks a lot fabrice,

we are also facing exactly the same proble(multiple bonding in rhel 4.7).
Let me try with the solution provided by you,and will update the comments.

Cheers,
satya

Reply

59 jacopo January 12, 2010 at 2:27 pm

Leave a comment… Well, this is more a question because I try to understand how to place
my actual IP address when I am using a client’s TOR network…

Reply

60 ap January 25, 2010 at 11:09 pm

Hello, I have few question
last week I playing with bonding.
At work I making file server, this is ubuntu server box with 2 nic’s also we have windows 7 clients with 2 nic as well & finally cisco switch 4500. Becourse we work with huge files, network speed is our main headache, so I try to aggregate network interface. I done it, but speed stay the same. I can utilize only 1 Gb connection (110 Mb /sec), but I want more!
Trying different conf, result the same. Even I make connection between win boxes without 802.3 ab cant have more than 110 Mb/sec. Now I think may be it is cisco misconfiguration?

Please tell me if some one got multiply speed bandwidth ?

Reply

61 GMB October 7, 2010 at 1:14 pm

@ap

LACP / 802.3ad only allows two end-points to max-out a single channel. Having 2x 1Gbps ports does not give you 2Gbps, but two 1Gbps devices can individually consume one each. LACP is also better placed for fail-over; pull a lead and communication continues. It does allow you to add many different speed lines too, so 2x 1Gpbs and 4x 100Mbps into one pool.

Also, are you sure you’re diferrentiating the difference between 1Gbps (1000Mbps) and 110MBps – note the lower / upper “B” for bits and Bytes. Therefore 110MB/s is around 880Mbps, or close to 1Gbps.

Reply

62 GMB October 7, 2010 at 1:22 pm

Just a thought as well – PCI Gigabit cards are limited to the bus speed of the host system and never quite give full Gigabit speeds. Better to get PCI-Express based NICs.

If it helps, here’s my Cisco C4507 configuration:

interface Port-channel11
description Proxy LACP 802.3ad
switchport
switchport access vlan 50
end

interface GigabitEthernet7/31
description proxy 1-2
switchport access vlan 50
channel-protocol lacp
channel-group 11 mode active
end

interface GigabitEthernet7/32
description proxy 2-2
switchport access vlan 50
channel-protocol lacp
channel-group 11 mode active
end

With Ubuntu configuration set with Bonding using mode=4 (for 802.3ad LACP)

https://help.ubuntu.com/community/LinkAggregation#mode=4%20%28802.3ad%29

Reply

63 Novajoseph March 5, 2010 at 10:06 am

Hi Seno,

u metioned above limited only by the number of network interfaces (NIC) Linux supports and/or the number of network cards you can place in your system

I have dought . . How many nw interface Redhat linux support and can bonding two bonding (bond0 ,bond1) …

i have 4 * 1 Gb in one PCI Xpress .How can i bond it..

Reply

64 Novajoseph March 5, 2010 at 12:22 pm

Hi nixcraft ,

u metioned above limited only by the number of network interfaces (NIC) Linux supports and/or the number of network cards you can place in your system

I have dought . . How many nw interface Redhat linux support and can bonding two bonding (bond0 ,bond1) …

i have 4 * 1 Gb in one PCI Xpress .How can i bond it..

Reply

65 Tarik March 23, 2010 at 11:25 am

Mike Baysek said: “If you are considering teaming NICs, remember the limitations of the bus itself. For example, if you think putting 4 Gigabit cards in your PCI bus will get you more speed, you are sadly mistaken! PCI bus is limited to maximum of around 133 MB/s. Putting two gigabit NICs on PCI will be a waste of time. Better to have at least PCI-X, or even better, PCI-e before you think about teaming your adapters.”
As per Wikipedia article on conventional PCI, the capacity of the PCI bus is 133 MB/s (32-bit at 33 MHz), 266 MB/s (32-bit at 66 MHz or 64-bit at 33 MHz) and 533 MB/s (64-bit at 66 MHz). Accordingly, PCI 32-bit at 66 MHZ or 64-bit at 33 MHz would only be saturated with 3 NICs. 64-bit at 66 MHz would take 7 NICs to be completely saturated.

Reply

66 Xarses March 30, 2010 at 6:51 am

An additional note is that while there are certain bus limitations as Tarik noted there are constraints of the bonding algorithm. Most notably is that in most configurations such as those mentioned here the maximum speed of a single connection is determined by the speed of the individual interface in the bonded group. In an example of 2 x 1Gbit adapters the maximum throughput for a single connection is 1Gbit/s. As I understand the limitation is that the Ethernet frames for any given conversation will not be moved to another interface unless that interface is down. The intent of this design is to prevent out of order frames which would ruin your day create performance issues on the end point that would make the extra performance moot. Not to mention that most security software and network devices would drop excessive amounts of this kind of traffic all together. If you want to use the full capability of the two links in our example you would have to devise a method to transfer the content using 2 or more conversations. IE cut the data into two groups and re-assemble it on the other side. Otherwise you would be better off using 10Gbit or other higher performance interfaces.

Reply

67 ashis Sharma April 2, 2010 at 6:56 pm

Really, very helpfull,

Thanks

Reply

68 Frank May 7, 2010 at 5:04 pm

How can we install mrtg to monitor the traffic comming on bond0

Reply

69 Hammad July 13, 2010 at 10:15 am

do i have to moidfy the networrk marsk too along with my ip address??

Reply

70 nick July 16, 2010 at 12:14 pm

Thanks nixcraft, always…
nice article, just need to add GATEWAY in bond0 file and voila!!

Reply

71 Garfield August 4, 2010 at 12:45 pm

I wonder to konw whether it support mutiple interface bond in IPv6? Can we receive a flow with multiple interfaces?

Reply

72 George September 14, 2010 at 1:38 pm

Great article – thanks for the info.

BTW – you may want to check your banner advertisers – My antivirus (nod32) is blocking some javascript content from your website as being malicious.

Reply

73 vinoth September 18, 2010 at 6:53 pm

Thanks for your quick comment about network bonding … you did such a nice job… keep it…….

Reply

74 Rahul October 15, 2010 at 10:43 am

Hi All,

this bonding automatically remove after the server shutdown or reboot

kindly check the same and let us know

Reply

75 GMB October 16, 2010 at 4:51 pm

@Rahul

If you’re bashing in commands and getting it working, then yes that remains in state so long as the OS is running. You need to edit your config files to make the bonding persistent through reboots. The link I posted above was for Ubuntu, but there should be a guide for your OS of choice.

Reply

76 tony November 1, 2010 at 8:06 pm

ifconfig does not show the inet addr for both interfaces “eth0 & eth2″ like the example given although bond0 looks good. Am I missing something ?

Reply

77 GMB November 9, 2010 at 7:45 pm

@tony
I guess it depends on the mode you’re using. The example given is using mode 6 “balance-alb” which can be used with basic switches. This therefore uses identical IPs and one active / one slave type mode.

Using mode 4 “802.3ad” would require an intelligent switch such as a Cisco and the switch configured for 802.3ad LACP mode also. This is an active / active mode and I guess (as I cannot remote into our server at work to check right now) will only give an IP on the bond interface since the other links are bonded in a truer sense.

Reply

78 sabry January 4, 2011 at 6:19 am

hello,

if let say i already bound this eth0 and eth1, and later maybe want it to be unbound again, so how can i do that?

thanks…

Reply

79 Rael January 20, 2011 at 3:19 am

I made a bond on OpenSuse 11.3 through the GUI. You just jump to the Network settings, and click on ADD. Choose ‘Bond’.
Then add to the bond the existing Nic-s – as many, as you have! I even added a fiber controller. Make sure the network adapters to be added don’t have any IP assigned, and also, in the Global options choose Never wake up! Then they’ll appear in the list of slave devices.
Cool, right! :)

Reply

80 paco sanchez January 31, 2011 at 4:35 am

I am surprised nobody has mentioned testing this “bond” I have the bond setup as described, however if i run ‘ifconfig eth0 down’, all of a sudden the link is not pingable, even though /proc/net/bonding/bond0 shows that it is up with eth1…

Another thought, does it seem a little odd that all the MAC addresses are the same?

Reply

81 GMB February 1, 2011 at 8:27 am

@paco sanchez

It would be my guess that the switch-side isn’t setup correctly if eth1 isn’t activating on fail-over (by you downing eth0 manually in this case). A real test would be to disconnect an Ethernet lead physically since that or a failed NIC / switch interface is more likely to cause the bond to continue communications over the remaining active links.

The MAC addresses become the same because that is how networks operate at the near-physical level. All NICs in a bond respond to requests to that address, though it is usually down to the switch to control the distribution of requests if it is an intellegent unit (Cisco et al) in which case it too will be configured for bonding using LACP or similar (and therefore offer round-robin / load-balancing type methods across the bonded links).

Reply

82 paco sanchez February 1, 2011 at 6:26 pm

Thanks GMB for your prompt reply. It makes sense what you are saying. In my setup, I have a VM running in VirtualBox with two vnics. I am not sure what the switch architecture is like for vbox but what I am looking for is adapter level redundancy. I know in the windows world, the “bond” takes on a virtual MAC which it uses on the network so as to avoid conflicts. If one adapter is downed, the other takes over at the physical layer using this “spoofed” MAC. I am looking for the same functionality in a Linux box, is this possible to acheive irrespective of my switch?

Reply

83 GMB February 5, 2011 at 11:54 pm

@paco sanchez
I might not of understood correctly, but are you trying to bond NICs within the guest VM? This would be pointless since virtual NICs are bridged to a single physical NIC.

I only use VirtualBox at home so I don’t use it in any fault-tolerant way, but at work I use XenServer that can bond physical NICs and link them to the guest VM as a (single) virtual NIC. The guest VM doesn’t need to manage fault tolerance since it will always have communication via the host’s fault tolerance.

For the mode setting you could try the “active / backup” mode, which doesn’t give you load balancing but does work with most basic layer 2 switches. If one link fails the other one simply takes over – making it fault tolerant, but only one link can be utilised for bandwidth at any time.

Interestingly, the MAC address is taken from the first network card in the bond unless explicitly set to something else during the configuration. That’s how you end up with one card’s MAC address spoofing for all of them.

Reply

84 paco sanchez February 6, 2011 at 2:15 am

GMB, you are correct, it is pointless inside a VM unless you are using it to learn. I am self-learning Linux (and loving every minute of it I might add) and just cannot afford to buy a bunch of physical dual-homed machines to try all this neat stuff out on. In theory, any VM (whether in virtualbox or another hypervisor) which is presented with multiple VNics ought to behave nearly similarly to a physical machine with two nics, am I incorrect in assuming this? I have verified that this works for Windows VM’s using “teaming”, I am simply trying to replicate this behavior in Linux. I have some update though.

Reply

85 GMB February 6, 2011 at 10:32 pm

@paco sanchez
Hello Paco, I’m understanding now. That should certainly work between guest VMs, being the virtual NIC driver doesn’t really go out to the physical NIC if the data remains within the host (so one guest doesn’t communicate to the switch and back in to the other guest, so to speak).

However, I’m not sure if the virtual switching would be LACP / 802.3ad aware or compatible, so you may well still be limited to active / backup mode.

Keep persevering in Linux though, the tide is changing and Microsoft isn’t the mainstay in the back office it once was. I’ve deployed Ubuntu and FreeBSD servers at my work for services such as MRTG and help desk system. I’ve been playing with both for over 15 years and I’m still learning!

Reply

86 Mike Smith April 11, 2014 at 10:12 am

Once eth0 is in a bond, you should not start doing ‘ifconfig eth0 down’ since eth0 must stay under the control of the bonding driver. The bonding driver uses the physical link state (the ethernet cable connected or not) to detemine a failed link, not the ethernet driver state (controlled by ifconfig). It is also worth reading https://www.kernel.org/doc/Documentation/networking/bonding.txt a few times!

Reply

87 George February 9, 2011 at 6:19 pm

same problem…
2 NICs, mode=1, 2 switches

Reply

88 William B. Peckham February 26, 2011 at 3:01 pm

nice!
Thank you for presenting this.

With gig NIC speed should not really be the issue. Load balancing helps, if the load justifies, but it is the failover capability that presents the real value.

Reply

89 BM Shukla March 13, 2011 at 10:55 am

Very good Note. I have used it in FC14 for 4 NICs. In think there exists one mistake. Please refer output sections. Both NICs acn not have MAC as 00:0C:29:C6:BE:59 .

Thanks

Reply

90 Arjun Kapoor April 25, 2011 at 6:10 pm

nice tutorial… please tell me how do i share this bond’s internet connection on the whole network via 3rd nic which will connect to the switch.

Reply

91 Arjun Kapoor April 25, 2011 at 6:12 pm

also please tell that if my 2 nics’ ip is dynamic the will it work….. please justify the bond’s ip address (is it the static ip for the network ?).
thank you

Reply

92 Tarun May 4, 2011 at 9:44 am

Thanks dear….
Good document…

Reply

93 Bret June 9, 2011 at 1:25 am

Hi Nixcraft,

I have got one linux server (CentOS 5 installed) with two NIC interface (eth0 & eth1) on it.

I want to configure server so that
– Only http and ssh traffic goes to/from eth0 interface (with IP-filtering) and
– Only https (external access) via eth1 interface. No other traffic allow on eth1 interface.

Is this possible to configure?

Thanks in advance,
Bret

Reply

94 nixCraft June 9, 2011 at 11:19 pm
  • eth1 – 202.54.1.1 bound https port to this ip
  • eth0 – 202.54.1.5 bound ssh and http port to this ip

Use plain old good IPtables to deny everything on eth1 interface except port 443 (https). Same goes for eth0 interface, block everything except port 80 (http), 22 (ssh) and other required port.

Reply

95 William Peckham June 9, 2011 at 3:07 pm

BRET
Interesting question, but WAY off topic. That has nothing to do with port binding.

Reply

96 Ethan Case July 5, 2011 at 8:14 am

LinuxTitli:
Thank you for this!! You made it super easy! (or maybe I’m finally learning linux)
To Bret, and others wanting to do Protocol Binding:
I’m pretty sure there is a method to do this in Linux, but you can also do this using a hardware-firewall. I use a Netgear SRX5308 – it’s a quad-WAN/LAN firewall. I have multiple static ip’s from different isp’s – using Protocol Binding on the firewall/router, I can set specific traffic to pass through certain WAN ips.

Reply

97 kumar July 6, 2011 at 3:52 pm

Hi
I am facing some sort of a peculiar issue. As per the step 1,2,3 i have done the bonding. Issue here is when i remove the eth1 physical port., the pinging will continue, but if i remove eth0 after eth1 is fixed back…GONE.. my network ping shows REQUEST TIME OUT. if i fix back the n/w cable no way still RTO.
Below is the h/w details for better understanding.
BL 460c G7, embedded 553i emulex 10G nic, connected to 1GB Ethernet Pass Thru, RHEL 5.6 x64.. Can anybody share your inputs to resolve this issue at the earliest…
Thanks … Vijay

Reply

98 kumar July 6, 2011 at 3:55 pm

Hi All
Another point to be added… Both the nics are connected to single Cisco 4500 switch and the round robin was also changed to fault tolerant (active-standby). Request your help
Reg Kumar

Reply

99 Mostafa July 8, 2011 at 5:43 pm

I have a PC and DSL connection. It`s OS is OpenSuse and router is from DLink. My PC can connect to Internet throw DSL and it is possible to ping it (it has static IP) but it is not possible to access SSH server from it.
It has 2 network interface, eth0 and dsl0. eth0 make a connection to the DLink router and dsl0 (which tack static ip) connect to the internet.
I need to access My PC from internet but it is not possible. Is it possible to solve my problem with bounded interfaces?
I`m new in linux, please help me. Thanks.

Reply

100 William Peckham July 10, 2011 at 5:17 pm

Mostafa: That is a router issue, you need to configure your router to pass outside traffic on port 22 ( and any other service ports you want to use for access from ‘outside’) to your server. It has nothing (IMOHO) to do with port bonding.

Reply

101 Nick July 22, 2011 at 10:09 am

Is it possible to apply this technique to a couple of USB data modems but use it to stream to another box – does the data stream have to be re-combined at the other end somehow to retain sync?
I’m trying to make a location video streaming solution.

Thanks

Reply

102 Vladimir November 11, 2012 at 12:10 am

I have the same problem task. Do you have a working example?

Reply

103 Vivek M Garg August 9, 2011 at 2:56 pm

Configuration was simple & worked perfectly well.
Got High throughput (HT) as well as High availability (HA) with the bond.
However after a reboot did not get HT or HA.

ifconfig is showing only bond0 & eth0.
eth1 is not visible in ifconfig. However it’s visible in ifconfig -a.
cat /proc/net/bonding/bond0 is also showing only bond0 & eth0.

What to do to sustain the bond after a reboot ? Please help !

Reply

104 Kevin Pham September 6, 2011 at 1:28 am

Thanks for the great doc! May I add a note please…

When the “modprobe bonding” command is issued, it may unload and reload the network modules and you’ll loose connection to the remote server and you may not be able to issue the next command. To make sure things work without loosing connection to the server, we can put both commands together in the same line

# modprobe bonding; /sbin/service network restart

Kevin

Reply

105 Arno September 10, 2011 at 10:29 pm

I see that is posible to bound more than 2 NIC`s with eachother, but my questions as following.
I have TP-Link TL-SG1016 (http://www.tp-link.com/uk/products/details/?categoryid=1588&model=TL-SG1016) and my mainboard is an Asus P5Q Premium that has 4 NIC`s onboard. i use it as a server with raid 5. I ask you, does the switch need also the trunk option to bound your NIC`s?

Reply

106 RK_RAJ October 24, 2011 at 12:23 pm

Hi friends,

By following this howto, I configured load balancing method of bonding with my 2 NICS in CentOS 5.5 X64 it works well without a single hitch, thanks for your tutorial

Reply

107 Netwall Expert December 8, 2011 at 1:33 pm

Dear all,
scenario is very simply,
I’ve Two ISPs with 256 nos of Real IPs for Lan side from both ISPs.
first ISP giving me 10 mbps of bandwidth and second ISP on 5 mbps of bandwidth.
I need to configure bonding network for my LAN users can access high bandwidth over 15 mbps of data means both bandwidth give data to user at full speed and Lan user also have a Real IP.
Is this scenario possible to create and How?
Current Setup
ISP 1 IP ETH0 – 202.54.15.2/255.255.255.252/DG 202.54.15.1
ISP 2 IP ETH2 – 202.54.16.2/255.255.255.252/DG 202.54.16.1
LAN Side ETH1 202.54.15.129/255.255.255.128
LAN Side ETH1:1 202.54.16.129/255.255.255.128

Please guide me how to create BOND of these king of Network

Thanks in Advance to all of you

Merry XMAS 2011

Reply

108 ravi December 22, 2011 at 10:47 pm

Excellent it worked !!!!

Reply

109 musa February 8, 2012 at 11:22 am

Hi everyone who are happy with this tutorial,

Would you kindly let me know if i should configure IPs (public) on my WAN interfaces of centos 5.5 if i need to bond them?…
this tutorial hasn’t explained it with some dummy IPs.
i am totally confused as which IPs to assign on WAN interfaces and which IP to assign to bond0 interface in such scenario.

Regards

Reply

110 Shankar February 15, 2012 at 6:29 pm

Hi,

Thanks for your excellent article.

I have a single application server which I need to connect to 2 Class 3 routers.

Can I use channel bonding for this, so that if once router goes down the application server is still reachable ?

If not, how can I do this ?

Regards,
Shankar

Reply

111 musa February 18, 2012 at 7:56 am

Any answer to my question plz.? m stuck :(

Reply

112 raj March 20, 2012 at 11:12 am

hello all,
i am facing very serious issue.i am working on mpc8315 processor board and i am using Linux 2.6.33 source. i have two Ethernet port in my board.when i port my Linux with filesystem , eth0 connected after eth1 down(using “ifconfig eth1 down”) and vice versa. i want to activate both. plz reply me

Reply

113 srinivas July 10, 2012 at 12:36 pm

Hi,

Thanks for posting.
I have face some regarding default gateway. I restarted the server or any interface down default gateway route is missing. Guys any help regarding this

Reply

114 jacky Ho July 17, 2012 at 6:24 am

To srinivas:
You must put following two command lines into /etc/rc.local when server boot
############################################################
/sbin/modprobe bonding
/sbin/service network restart
############################################################
load the bonding module, enter:
# modprobe bonding
Restart the networking service in order to bring up bond0 interface, enter:
# service network restart
############################################################

Reply

115 jake0391 July 19, 2012 at 2:40 pm

What would happen if you bond two ethernet ports on two different ethernet cards that had different firmware levels ?

# ethtool -i eth1
driver: netxen_nic
version: 4.0.75
firmware-version: 4.0.544
# ethtool -i eth5
driver: netxen_nic
version: 4.0.75
firmware-version: 4.0.555
bus-info: 0000:05:00.1

Reply

116 Mike Smith April 11, 2014 at 10:18 am

The bonding driver doesn’t care. You can bond two cards from two manufacturers if you want to. As long as the card drivers support ethtool and ifconfig then they should work (the bonding driver uses various ioctl calls to the ethernet drivers to do its thing).

Reply

117 William July 19, 2012 at 5:33 pm

Raj

If you haven’t got to the bottom of your problem try adding the MAC address on each controller. Look up HWADDR and add that to eth0 and eth1.

Thanks

Reply

118 Vikas July 26, 2012 at 4:17 am

Really very helpfull..thanks a lot for taking the time and posting this.

Reply

119 Mallik Kongara August 16, 2012 at 8:40 pm

Hi,
I just tried setting up NIC bonding in a OEL machine. I setup the bond0 interface in mode=1 a.k.a. fault-tolerant (active/passive) mode. According to the documentation, in this mode all of the traffic should be received and transmitted via the primary active interface. However, in reality, I see network traffic being received on both physical interfaces that have been bonded to bond0. Is this how it is supposed to be? Am I missing something here?
– Mallik

Reply

120 Mike Smith April 11, 2014 at 10:29 am

When the bonding is running, packets are only transmitted from the active ethernet interface. So if you are seeing pings back from a supposedly inactive ethernet interface, this means that you probably haven’t bonded the drivers properly or in the correct mode. The default bonding mode is balance-rr (mode 0) which would probably behave as you are seeing.

You can check the mode by typing the following at the terminal:-

cat /proc/net/bonding/bond0

to see what the bonding driver thinks it is doing.

You can set the mode by typing the following at the terminal:-

echo active-backup > /sys/class/net/bond0/bonding/mode

If you want to try adding an ethernet interface to the bond, type:-

echo +ethN > /sys/class/net/bond0/bonding/slaves

To remove an interface type:-

echo -ethN > /sys/class/net/bond0/bonding/slaves

Reply

121 Imran Haider August 28, 2012 at 5:41 am

Really Very Helpful

Reply

122 Banerjee August 30, 2012 at 5:23 pm

I appreciate your blog.
Thanks a lot for putting this up.
I need to configure a high availability network using RHEL-5 Which mode should I prefer(higher throughput is desirable,not mandatory)? active-backup or balance-alb?
How do I remove the bonding?

Reply

123 BrianBlaze September 28, 2012 at 1:50 pm

I noticed that for bond 0 eth0 and eth2 are both masters… what is the result of that?

Reply

124 Kumar Amit September 29, 2012 at 9:18 am

nice stuff :)

Reply

125 Ankzz January 15, 2013 at 9:41 am

Check this utility “getBondStatus” to check the bonded interface capability :

http://bondedinterface.blogspot.in/

Utility is at link :
https://github.com/ankkzz/getBondStatus/

Reply

126 Jason Barnett February 13, 2013 at 2:17 am

Not sure if this has been mentioned before, but Redhat specifically says, “Do not place parameters for the bonding kernel module in the /etc/modprobe.conf”

Reference: redhat.com

Reply

127 Angelo March 7, 2013 at 6:37 pm

Hey guys
did anyone tried to create a set of vlan interfaces on top of a bond device ?

Reply

128 Mac October 17, 2013 at 10:36 pm

Bonding 4 nics is the same procedures in RHEL6 variants you have to create the /etc/modprobe.d/bonding.conf file.
Also, verify the bonding type mode, along with the correct syntax in the ifcfg-bond0 ect… will have the bonding options.

Reply

129 Laith Al Obaidy February 27, 2014 at 8:24 am

I have done the same scenario, but I got a Flapping in the switches.

Reply

130 saravana kumar April 14, 2014 at 4:25 pm

Hi,
Thanks for your documentation, I have followed your steps, when i try to bring eth0, am unable to reach my server. I am trying this in virutalbox centos6.3. Please help me to fix the problem.

Please find the pastebin link. http://paste.ubuntu.com/7250504/

Reply

131 Eric B October 7, 2014 at 7:10 pm

Try the following for testing the bonding and failover:

Instead of using the one shot command: cat /proc/net/bonding/bond0
Why not use the following to monitor realtime condition changes.

watch -n 1 ‘cat /proc/net.bonding/bond0′

Change the -n 1 value to change the refreash from 1 sec..

you can manually make condition changes using “ifdown eth0″ and “ifup eth0″ while running a continuose ping will show you when the failover occurs if any data streaming is stopped. Apply this also to the other partner NIC. Control C to exit.

Reply

Leave a Comment

Tagged as: , , , , , , , , , , , , , , , , , , , , , , , , ,

Previous post:

Next post: