Linux: Configuring RX POLLING (NAPI)

by on November 27, 2009 · 1 comment· LAST UPDATED November 27, 2009

in , ,

My CentOS / RHEL based server is configured with Intel PRO/1000 network interface cards. NAPI (Rx polling mode) is supported in the e1000 driver. I've multiple CPUs. How do I configure NAPI, decreasing interrupts and improve overall server network performance?

NAPI is enabled or disabled based on the configuration of the kernel. e1000 driver does supports NAPI.

Enable NAPI

Downolad the latest driver version by visting the following url:

  1. Linux kernel driver for the Intel(R) PRO/100 Ethernet devices, Intel(R) PRO/1000 gigabit Ethernet devices, and Intel(R) PRO/10GbE devices.

To enable NAPI, compile the driver module, passing in a configuration option:
make CFLAGS_EXTRA=-DE1000_NAPI install
Once done simply install new drivers.

Decreasing Interrupts

If you've multiprocessor systems, consider binding the interrupts of the network interface cards to a physical CPU to gain additional performance. Find out IRQ of of the network card, run:
# ifconfig eth0
OR
# ifconfig eth0 | grep -i Interrupt
Sample outputs:

Interrupt:179

To set the smp_affinity i.e. bound interrupt 179 of eth0 to the third processor in the system, enter:
# echo 03 > /proc/irq/179/smp_affinity
Add above command to /etc/rc.local. See Intel e1000 documentation for more information (Kernel v2.6.26 or above turns on NAPI support by default). Broadcom tg3 drives also support NAPI and the latest version comes with built in NAPI support.

TwitterFacebookGoogle+PDF versionFound an error/typo on this page? Help us!

{ 1 comment… read it below or add one }

1 Jeff Schroeder November 28, 2009 at 3:50 am

This is not the proper way to do this. First off, what if you have an ssd drive putting most of the writes on cpu core 3? What if you have some virtualizaton solution driving the interrupts on core3 sky high? This will signifigantly lower performance if you do it incorrectly. How do you know? Blindly pinning a nic to some random CPU core is a BAD idea unless you actually know what you’re doing. You should first watch the interrupts:
watch -n 1 cat /proc/interrupts

Then you can find a free cpu core to pin the nic too. It will be the obvious one where the number of interrupts is not going up as quickly or isn’t as high as the other ones. You should also look at running irqbalance unless you actually know what you’re doing.

This might also help if you want to script this stuff:
intr=$(ifconfig eth0 | awk -F: ‘/Interrupt:/{print $NF}’)

Additionally, take a look at network card coalescing. ethtool -c eth0 to see the current coalescing settings for eth0 and you can change them with ethtool -C. That can change the number of interrupts drastically from batching interrupts for large numbers of packets or a single interrupt for packet. The tradeoff is latency vs throughput. Pick one or the other.


My linux blog

Reply

Leave a Comment

Tagged as: , , , , , , , , , , , , , ,

Previous Faq:

Next Faq: