Q. Jumbo frames are Ethernet frames with more than 1500 bytes of payload MTU. Does Linux support jumbo frames? If so how do I set frames to 9000 bytes under Linux operating systems?
A. Most modern Linux distros (read as Linux Kernel 2.6.17+) does support frames larger than 1500 bytes. This can improve the performance. First, make sure your network driver supports custom MTU. Second you need to have a compatible gigabit NIC and switch (such as Cisco Catalyst 4000/4500 Switches with Supervisor III or Supervisor IV ) that is jumbo frame clean. If you are not sure about requirements, please refer to your product documentation.
Jumbo frames can reduce server overhead such as a big decrease in CPU usage when transferring larger file. Also you should see some increase in network throughput.
Configure eth0 interface for Jumbo Frames
WARNING! These examples may not work with your setup as it depends upon a compatible network gear such gigabit Ethernet switches and gigabit Ethernet network interface cardsSimply type the following command at a shell prompt to set new MTU (Jumbo Frames):
# ifconfig eth0 mtu 9000
Make changes permanent
Edit the network configuration file for eth0 interface – for example, /etc/sysconfig/network-script/ifcfg-eth0 (CentOS / RHEL / Fedora Linux):
# vi /etc/sysconfig/network-script/ifcfg-eth0
Append the following configuration directive, which specifies the size of the frame in bytes:
MTU 9000
A note about Debian / Ubuntu Linux user:
Debian / Ubuntu Linux user should add MTU=9000 to /etc/network/interfaces configuration file.
Close and save the file. Restart the Interface eth0:
# /etc/init.d/networking restart
To confirm the MTU used between two specific devices. use ip command as follows:
# ip route get {IP-address}
# ip route get 192.168.1.1
Output:
192.168.1.1 dev eth0 src 192.168.1.100
cache mtu 9000 advmss 1460 hoplimit 64Application Protocol Tunning
You may need to tune the application / network protocol such as NFS and SMB to take advantage of Jumbo Frames.
Updated for accuracy.
19 comment
Hi, nice article
just one (1) error; for debian/ubuntu in /etc/network/interfaces, it’s mtu 9000 (mtu(space)9000) *NOT* mtu=9000 (this is only for redhat linuxes)!
THANK YOU!! You saved the whole point of this article, I was looking specifically for jumbo frames for ubuntu
Actually that only works if your statically setting the IP/etc… for that interface. If you’re using dhcp you have to set the MTU after the card is configured otherwise it will be overwritten by whatever the dhcp server passes it. To get around this I setup mine up like this:
auto eth0
iface eth0 inet dhcp
post-up /sbin/ifconfig eth0 mtu 7000
Thanks for the article. I have a second ethernet card that’s a gigabit card and I tried running ifconfig eth1 mtu 9000 and it gives me an error “SIOCSIFMTU: Invalid argument”. Any idea why this is happening? Also, modifying /etc/network/interfaces didn’t seem to help either.
Chris; I just had the same thing happen on Hardy and a quick Google shows that we are not alone. It must be outside the range that our drivers/hw allow for this chipset, just taking a huge swag. I really need this to work too. FWIW:
processor : 0
vendor_id : AuthenticAMD
cpu family : 15
model : 104
model name : AMD Turion(tm) 64 X2 Mobile Technology TL-60
nVidia MCP67 Ethernet
Linux jeff-laptop 2.6.24-19-generic #1 SMP Wed Jun 18 14:43:41 UTC 2008 i686 GNU/Linux
Will get to the bottom of this since I just spent the previous weekend rewiring the house with cat6, upgraded all routers, etc….
Jeff
Follow-on to my last but just got new information. Chris, open a term and do a ‘sudo ethtool eth0’ or whatever your actual device is. Amongst other useful information there was this (and yes I do have a gigabit ethernet card):
Settings for eth0:
Supported ports: [ MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Advertised auto-negotiation: Yes
Speed: 100Mb/s
Sorry if that came out dodgy but the thing to note is my driver only thinks I can do 100M and as such, I would assume it would be hard for the above command to work. Well, there is a new kernel out too so I might as well grab the drivers and make a party of it…
Jeff,
Thanks for your reply. It’s strange how you have a gigabit card, yet your limit was only 100Mb/s. Perhaps you had another one, eth1? I know mine correctly has a max limit of 1000Mb/s, but somehow I couldn’t get it to work initially. I found out my card (Netgear GA311) had a max MTU of 7500, but setting that didn’t work. When I did a little bit more googling, I found out that someone who had the same card set the MTU to 7200. Once I did that, it worked for me. So I guess your initial theory was right in that the number was out of range of the max MTU.
Hi Vivek,
What is the downside for setting higher MTU?
How would I know the proper value for MTU for my server?
I am using rhel 4 AS (2.6.9-42) , BroadCom Corporation NetXtreme II BCM5708 Gigabit Ethernet network Card and switch is 3Com® Baseline Switch 2948-SFP Plus.
Jay
Can’t set mtu > 1500 on Ubuntu 9.10. This sucks! I am trying to find tweaks to fool OS but no luck yet. It works in windows very well > 9000kb jumbo size frames!
sudo ethtool eth0
gives
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: g
Wake-on: g
Current message level: 0x000000ff (255)
Link detected: no
For the Fedora/RH/CentOS folks make sure there is an = in the line
MTU=9000
Sometimes it happens to be only a matter of mms, but it all depands on the module/driver rivision of your NIC. What version of the NDIS driver you said you are using? [email protected]
Help me!!! :)
I try change my MTU to 9000, but a receive this error (SIOCSIFMTU: Argumento inválido) . however i can change to 7004.
My configuration:
[[email protected] rafael]# ifconfig eth0
eth0 Link encap:Ethernet Endereço de HW 00:26:9E:21:F2:D3
inet end.: 10.10.10.106 Bcast:10.10.10.255 Masc:255.255.255.0
endereço inet6: fe80::226:9eff:fe21:f2d3/64 Escopo:Link
UP BROADCASTRUNNING MULTICAST MTU:7004 Métrica:1
RX packets:21212 errors:0 dropped:0 overruns:0 frame:0
TX packets:15319 errors:0 dropped:0 overruns:0 carrier:0
colisões:0 txqueuelen:1000
RX bytes:24128672 (23.0 MiB) TX bytes:2527859 (2.4 MiB)
IRQ:31 Endereço de E/S:0xa000
[[email protected] rafael]# ethtool eth0
Settings for eth0:
Supported ports: [ TP MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: MII
PHYAD: 0
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: pumbg
Wake-on: g
Current message level: 0x00000033 (51)
Link detected: yes
[[email protected] rafael]# ifconfig eth0 mtu 9000
SIOCSIFMTU: Argumento inválido
Thanks.
Rafael
Rafael, virgoptrex, maybe your driver is not supporting yet big MTUs. You can learn the driver you’re using with:
ethtool -i XXX
Example:
[email protected]# ethtool -i eth1
driver: igb
version: 2.1.1
firmware-version: 1.7-2
bus-info: 0000:05:00.1
FYI, some of the interfaces/chipsets do not support jumbo frames such as Intel’s 82573V.
ref:
http://ark.intel.com/products/26551/Intel-82573V-Gigabit-Ethernet-Controller
Hey all, I just noticed that a more helpful message is printed in the system log.
dmesg:
e1000: eth0: e1000_change_mtu: Jumbo Frames not supported.
Strangely the Supermicro board I have has dual Ethernet and the secondary interface supports Jumbo frames, but not the primary!
lspci:
04:00.0 Ethernet controller: Intel Corporation 82573E Gigabit Ethernet Controller
05:00.0 Ethernet controller: Intel Corporation 82573L Gigabit Ethernet Controller
The path in the article:
# vi /etc/sysconfig/network-script/ifcfg-eth0
Should be (scriptS, plural for CentOS/rhe):
# vi /etc/sysconfig/network-scripts/ifcfg-eth0
Pretty sure the advmss value should say 8960…
1460 is the max seg size for MTU 1500…
can is set MTU 8192 to 1G nic card??
Speed: 1000Mb/s
Duplex: Full
I also want to configure two servers with a cross-over Cat6 cable, but still un-successful.
I am testing by sending big packets, as follows:
# ping -s 8972 192.168.0.20
#demsg
tg3 0000:03:00.1: eth1: Link is up at 1000 Mbps, full duplex
tg3 0000:03:00.1: eth1: Flow control is off for TX and off for RX
tg3 0000:03:00.1: eth1: EEE is disabled