Linux Increasing The Transmit Queue Length (txqueuelen)

Posted on in Categories , , , , , , , , , last updated October 22, 2009

How do I set txqueuelen (the length of the transmit queue) length of the network card device under Linux? How do I set it for a high latency and/or a high speed network for bulk transfer from disturbing interactive traffic like ssh too much?

You can set the length of the transmit queue of the device using /sbin/ifconfig command as follows:

  1. Small value for slower devices with a high latency like modem links and ISDN.
  2. High value is recommend for server connected over the high-speed Internet connections that perform large data transfers.

Increasing The txqueuelen

Increase the txqueuelength parameter to a value between 1000 and 20000 per interface:

ifconfig ${interface} txqueuelen ${size}
ifconfig eth1 txqueuelen 10000
ifconfig eth0 txqueuelen 5000

Setting the txqueuelen permanently

Edit /etc/rc.locale, enter:
vi /etc/rc.local
Append the following setting per interface:

/sbin/ifconfig eth1 txqueuelen 10000
/sbin/ifconfig eth0 txqueuelen 10000

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin and a trainer for the Linux operating system/Unix shell scripting. He has worked with global clients and in various industries, including IT, education, defense and space research, and the nonprofit sector. Follow him on Twitter, Facebook, Google+.

13 comment

  1. Thx for posting this howto for setting txqueuelen.

    It should be noted however that increasing the txqueuelen value is a sure fire way to oversaturate your network interface buffers which may cause excessive jitter and DNS resolve timeouts. The default Linux txqueuelen value of 1000 is already way to high, and reducing this value to some much smaller number will reduce the likelihood of your network interface buffers being oversaturated. Personally I have had good results setting the value to 200.

    Basically TCP must be able to have packets dropped quickly in order for the server to know that there is network congestion occurring. The size of the value determines how much buffering takes place(where # of packets translates to number of seconds of data being buffered). When this value is too high the feedback mechanism, which forms the basis for the network congestion avoidance algorithms in TCP, fails, meaning instead of packets being dropped thereby signalling that there is congestion. When the server is unaware of congestion due to excessive buffering the inevitable retransmission of packets only occurs once the problem has gotten too severe- and thus gigantic latencies are introduced, latencies equivalent to the number of seconds buffered according to the value set in txqueuelen. So the smaller the txqueuelen value, the better are the chances that TCP’s congestion avoidance algorithms can do their job-hence resulting in higher transfer speeds, and the less likely that the buffers will be overfilled leading to gigantic timeouts when retransmission inevitably occurs. Google Jim Gettys and bufferbloat for a better explanation.

  2. Smaller will make your TCP stack more responsive. TCP does plenty of buffering for you already.

    Unmanaged FIFO buffers greater than, say, 32, are bad. You can use bigger buffers, but you should then use a qdisc of some sort to balance your flows. I can’t imagine a valid reason for any level of FIFO buffering at present.

    Try also enabling ECN and SACK.

    The previous poster – while otherwise very correct – also missed using ethtool to reduce your dma tx queues.

    See also:

  3. I find a slightly cleaner way to make this permanent is to vi /etc/network/interfaces and add the following line below the interface you want to change:

    up ifconfig $IFACE txqueuelen 10000

    The up allows any following command to be executed when the interface is brought up. This way the line is in the same place as the rest of the interface configuration.

  4. Hi
    Thanks for your good instruction
    May you give me more information about reasons of “PACKET DROP” ?
    If NIC gets one corrupted packet, is number increased? or system does not count corrupted packet at all?

    Thanks in advance

  5. Not sure, remember that the device is always handled by a CPU. It is useful to set the txqueuelen to small values for slower devices with a high latency (modem links, ISDN) to prevent fast bulk transfers from disturbing interactive traffic. Therefore careful tuning is required to ensure that the sizes of the queues are optimal for your network connection. The default of 100 is inadequate for long distance, high throughput pipes. Also you have to consider the receiver side, we have a similar queue for incoming packets. This queue will build up in size when an interface receives packets faster than the kernel can process them. If this queue is too small (default is 1000), we will begin to loose packets at the receiver, rather than on the network. I have doing more and more testing and personally i have more better performance and lowest cpu load on stress server quad core and giga ethernet at about 4650mi distance from me with this 2 setting:
    ifconfig eth0 txqueuelen 2000
    echo 2000 > /proc/sys/net/core/netdev_max_backlog

  6. when i try
    ifconfig ${interface} txqueuelen ${size}
    ifconfig eth1 txqueuelen 10000
    ifconfig eth0 txqueuelen 5000

    i get this:
    SIOCSIFTXQLEN: No such device
    My question if i add lines

    /sbin/ifconfig eth1 txqueuelen 10000
    /sbin/ifconfig eth0 txqueuelen 10000
    in rc.local its enoguht for increase it?

  7. This way you can only change the txqueuelen temporarily , /etc/rc.local runs at boot time, but when you restart your interface it will again set the txqueuelen value to default. here is the permanent way of fixing it. in below example i am setting txqueuelen to 5000 permanently. This value will remain same even after reboot of server or just interface restart/reset.

    edit /etc/sysconfig/network-script/ifup and place your command ” ifconfig eth2 txqueuelen 5000″ in beginning of the file ,just above line “unset WINDOW # defined by screen, conflicts with our usage”

Leave a Comment