How to test the network speed/throughput between two Linux servers

I have setup total four servers in cloud based data center. They run either CentOS or Ubuntu Linux. I need to make sure clustered file system performance is reasonable as my servers shared with other users. How do I check the speed between the two Linux servers using command line options for private LAN/VLAN?

You can test the network speed/throughput between Ubuntu/CentOS/Debian/Fedora Linux or Unix box using the iperf command. Iperf commands show info about bandwidth, delay, jitter, and datagram loss. It is a tool for performing network throughput measurements. It can test either TCP or UDP throughput. To perform an iperf test the user must establish both a server (to discard traffic) and a client (to generate traffic). This page explains how to test the network speed/throughput between two Linux servers.
Tutorial details
Difficulty level Easy
Root privileges Yes
Requirements Linux with iperf
Est. reading time 8 mintues

My sample set up to test the network speed/throughput between two Linux servers

Fig.01 Test network speed between two Linux computers

Fig.01 Test network speed between two Linux computers

Step 1 – Installation

You need to install iperf as per your Linux distro version. There are two versions of iperf:

  1. iperf
  2. iperf3

The iperf/iperf3 works on Linux, Unix, Windows, *BSD and other operating systems.

Install iperf on a Ubuntu/Debian Linux server

Type the following apt-get command/apt command to install iperf on both servers:
$ sudo apt-get update
$ sudo apt-get install iperf

Here is what we see on Ubuntu 20.04 LTS system:

[sudo] password for vivek: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  iperf
0 upgraded, 1 newly installed, 0 to remove and 23 not upgraded.
Need to get 76.5 kB of archives.
After this operation, 213 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu focal/universe amd64 iperf amd64 2.0.13+dfsg1-1build1 [76.5 kB]
Fetched 76.5 kB in 1s (70.5 kB/s)                    
Selecting previously unselected package iperf.
(Reading database ... 344795 files and directories currently installed.)
Preparing to unpack .../iperf_2.0.13+dfsg1-1build1_amd64.deb ...
Unpacking iperf (2.0.13+dfsg1-1build1) ...
Setting up iperf (2.0.13+dfsg1-1build1) ...
Processing triggers for man-db (2.9.1-1) ...

A note about installing iperf3 on a Debian/Ubuntu Linux

It is also possible to install iperf3 (latest version) as follows:
$ sudo apt-get install iperf3
Outputs:

Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  libiperf0
The following NEW packages will be installed:
  iperf3 libiperf0
0 upgraded, 2 newly installed, 0 to remove and 23 not upgraded.
Need to get 86.2 kB of archives.
After this operation, 294 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://archive.ubuntu.com/ubuntu focal/universe amd64 libiperf0 amd64 3.7-3 [72.0 kB]
Get:2 http://archive.ubuntu.com/ubuntu focal/universe amd64 iperf3 amd64 3.7-3 [14.2 kB]
Fetched 86.2 kB in 1s (97.4 kB/s) 
Selecting previously unselected package libiperf0:amd64.
(Reading database ... 344807 files and directories currently installed.)
Preparing to unpack .../libiperf0_3.7-3_amd64.deb ...
Unpacking libiperf0:amd64 (3.7-3) ...
Selecting previously unselected package iperf3.
Preparing to unpack .../iperf3_3.7-3_amd64.deb ...
Unpacking iperf3 (3.7-3) ...
Setting up libiperf0:amd64 (3.7-3) ...
Setting up iperf3 (3.7-3) ...
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for libc-bin (2.31-0ubuntu9.2) ...

Installing iperf on a Fedora latest/RHEL/CentOS 8 Linux server

Type the following dnf command to install iperf on both servers:
$ sudo dnf install iperf

A note about installing iperf3 on a CentOS/RHEL/SL/Oracle Linux

Type the following command:
$ sudo dnf install iperf3

Install iperf on a CentOS/RHEL/Oracle/Scientific Linux server

Type the following yum command to install iperf on both servers (first turn on EPEL repo for CentOS/RHEL 7.x):
$ sudo yum install iperf
If you are using the latest version of Fedora and CentOS/RHEL 8.x try the dnf command as follows:
$ sudo dnf install iperf3

How to test the network speed throughput between two Linux servers

Installing iperf3 on a CentOS or RHEL 8.x server

A note about installing iperf3 on a CentOS/RHEL/SL/Oracle Linux

Type the following command:
$ sudo yum install iperf3

Step 2 – Start an iperf server on serverA (server mode)

Type the following command to run iperf in server mode:
iperf -s
OR
iperf3 -s
Sample outputs:

------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------

Please note down the TCP port # 5001.

A note about firewall config

Please note that you must open TCP port 5001 on the serverA. See how to use ufw to open a port on Ubuntu/Debian Linux:
$ sudo ufw allow from 192.168.149.0/24 to 192.168.149.69 port 5001 proto tcp
CentOS/RHEL/Fedora users need to the port using FirewallD. See how to set up a firewall using FirewallD on CentOS 8:
sudo firewall-cmd --zone=public --add-port=5001/tcp --permanent
It is also possible to change the port by passing the -p option as follows (e.g. open and use TCP port 2323):
iperf -s -p 2323
iperf3 -s -p 4242

-----------------------------------------------------------
Server listening on 4242
-----------------------------------------------------------

Step 3 – Start an iperf server on serverB (client mode)

Type the following command to run in client mode, connecting to 192.168.149.69:
iperf -c {ip-address-of-serverA-here}
iperf -c {ip-address-of-serverA-here} -p {tcp-port}
iperf -c 192.168.149.69 -p 2323
iperf -c 192.168.149.69

OR
iperf3 -c {ip-address-of-serverA-here}
iperf3 -c {ip-address-of-serverA-here} -p {tcp-port}
iperf3 -c 192.168.149.69 -p 2323
iperf3 -c 192.168.149.69

Sample outputs:

Fig.02: Linux network speed test result

Fig.02: Linux network speed test result

How to use UDP rather than TCP

On the server side:
iperf -s -u
On the client side:
iperf -c 192.168.149.69 -u

See also: How To Install Speedtest-cli On a CentOS / RHEL / Scientific / Fedora Linux To Check Internet Speed From the Bash Shelll

How to set number of parallel client threads to run (client mode options)

Pass the -P option:
iperf -c 192.168.149.69 -P 3
You can also set time in seconds to transmit for (default 10 secs) as follows:
iperf -c 192.168.149.69 -P 3 -t 30
Sample outputs:

------------------------------------------------------------
Client connecting to 192.168.149.69, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  5] local 192.168.181.40 port 54198 connected with 192.168.149.69 port 5001
[  3] local 192.168.181.40 port 54194 connected with 192.168.149.69 port 5001
[  4] local 192.168.181.40 port 54196 connected with 192.168.149.69 port 5001
[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-30.0 sec  2.26 GBytes   647 Mbits/sec
[  3]  0.0-30.0 sec  2.89 GBytes   828 Mbits/sec
[  4]  0.0-30.5 sec  1.91 GBytes   539 Mbits/sec
[SUM]  0.0-30.5 sec  7.07 GBytes  1.99 Gbits/sec

Here is output from iperf3:
## on the server side ##
iperf3 -s
## on the client side ##
iperf3 -c 192.168.149.69

Sample outputs:

Connecting to host 192.168.149.69, port 5201
[  4] local 192.168.181.40 port 55522 connected to 192.168.149.69 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   235 MBytes  1.97 Gbits/sec   32    609 KBytes       
[  4]   1.00-2.00   sec   230 MBytes  1.93 Gbits/sec   21    670 KBytes       
[  4]   2.00-3.00   sec   159 MBytes  1.33 Gbits/sec   19    635 KBytes       
[  4]   3.00-4.00   sec   235 MBytes  1.97 Gbits/sec    7    686 KBytes       
[  4]   4.00-5.00   sec   229 MBytes  1.92 Gbits/sec   86    757 KBytes       
[  4]   5.00-6.00   sec   206 MBytes  1.73 Gbits/sec    6    537 KBytes       
[  4]   6.00-7.00   sec   210 MBytes  1.76 Gbits/sec    5    556 KBytes       
[  4]   7.00-8.00   sec   235 MBytes  1.97 Gbits/sec   10    617 KBytes       
[  4]   8.00-9.00   sec   245 MBytes  2.06 Gbits/sec   11    670 KBytes       
[  4]   9.00-10.00  sec   234 MBytes  1.96 Gbits/sec   10    713 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  2.17 GBytes  1.86 Gbits/sec  207             sender
[  4]   0.00-10.00  sec  2.16 GBytes  1.86 Gbits/sec                  receiver

Getting help is easy

Type the following command:
$ man iperf
$ man iperf3
$ iperf --help
$ iperf3 --help

Server or Client:
  -p, --port      #         server port to listen on/connect to
  -f, --format   [kmgtKMGT] format to report: Kbits, Mbits, Gbits, Tbits
  -i, --interval  #         seconds between periodic throughput reports
  -F, --file name           xmit/recv the specified file
  -A, --affinity n/n,m      set CPU affinity
  -B, --bind      <host>    bind to the interface associated with the address <host>
  -V, --verbose             more detailed output
  -J, --json                output in JSON format
  --logfile f               send output to a log file
  --forceflush              force flushing output at every interval
  -d, --debug               emit debugging output
  -v, --version             show version information and quit
  -h, --help                show this message and quit
Server specific:
  -s, --server              run in server mode
  -D, --daemon              run the server as a daemon
  -I, --pidfile file        write PID file
  -1, --one-off             handle one client connection then exit
  --rsa-private-key-path    path to the RSA private key used to decrypt
                            authentication credentials
  --authorized-users-path   path to the configuration file containing user
                            credentials
Client specific:
  -c, --client    <host>    run in client mode, connecting to <host>
  --sctp                    use SCTP rather than TCP
  -X, --xbind <name>        bind SCTP association to links
  --nstreams      #         number of SCTP streams
  -u, --udp                 use UDP rather than TCP
  --connect-timeout #       timeout for control connection setup (ms)
  -b, --bitrate #[KMG][/#]  target bitrate in bits/sec (0 for unlimited)
                            (default 1 Mbit/sec for UDP, unlimited for TCP)
                            (optional slash and packet count for burst mode)
  --pacing-timer #[KMG]     set the timing for pacing, in microseconds (default 1000)
  --fq-rate #[KMG]          enable fair-queuing based socket pacing in
                            bits/sec (Linux only)
  -t, --time      #         time in seconds to transmit for (default 10 secs)
  -n, --bytes     #[KMG]    number of bytes to transmit (instead of -t)
  -k, --blockcount #[KMG]   number of blocks (packets) to transmit (instead of -t or -n)
  -l, --length    #[KMG]    length of buffer to read or write
                            (default 128 KB for TCP, dynamic or 1460 for UDP)
  --cport         <port>    bind to a specific client port (TCP and UDP, default: ephemeral port)
  -P, --parallel  #         number of parallel client streams to run
  -R, --reverse             run in reverse mode (server sends, client receives)
  --bidir                   run in bidirectional mode.
                            Client and server send and receive data.
  -w, --window    #[KMG]    set window size / socket buffer size
  -C, --congestion <algo>   set TCP congestion control algorithm (Linux and FreeBSD only)
  -M, --set-mss   #         set TCP/SCTP maximum segment size (MTU - 40 bytes)
  -N, --no-delay            set TCP/SCTP no delay, disabling Nagle's Algorithm
  -4, --version4            only use IPv4
  -6, --version6            only use IPv6
  -S, --tos N               set the IP type of service, 0-255.
                            The usual prefixes for octal and hex can be used,
                            i.e. 52, 064 and 0x34 all specify the same value.
  --dscp N or --dscp val    set the IP dscp value, either 0-63 or symbolic.
                            Numeric values can be specified in decimal,
                            octal and hex (see --tos above).
  -L, --flowlabel N         set the IPv6 flow label (only supported on Linux)
  -Z, --zerocopy            use a 'zero copy' method of sending data
  -O, --omit N              omit the first n seconds
  -T, --title str           prefix every output line with this string
  --extra-data str          data string to include in client and server JSON
  --get-server-output       get results from server
  --udp-counters-64bit      use 64-bit counters in UDP test packets
  --repeating-payload       use repeating pattern in payload, instead of
                            randomized payload (like in iperf2)
  --username                username for authentication
  --rsa-public-key-path     path to the RSA public key used to encrypt
                            authentication credentials
 
[KMG] indicates options that support a K/M/G suffix for kilo-, mega-, or giga-
 
iperf3 homepage at: https://software.es.net/iperf/
Report bugs to:     https://github.com/esnet/iperf

Summing up

This page explained how to perform network throughput tests between two Linux or Unix server using iperf3/iperf tool. See iperf documenation and iperf3 source code online for further information.


🐧 Get the latest tutorials on Linux, Open Source & DevOps via RSS feed or Weekly email newsletter.

🐧 5 comments so far... add one

CategoryList of Unix and Linux commands
Disk space analyzersncdu pydf
File Managementcat
FirewallAlpine Awall CentOS 8 OpenSUSE RHEL 8 Ubuntu 16.04 Ubuntu 18.04 Ubuntu 20.04
Network UtilitiesNetHogs dig host ip nmap
OpenVPNCentOS 7 CentOS 8 Debian 10 Debian 8/9 Ubuntu 18.04 Ubuntu 20.04
Package Managerapk apt
Processes Managementbg chroot cron disown fg jobs killall kill pidof pstree pwdx time
Searchinggrep whereis which
User Informationgroups id lastcomm last lid/libuser-lid logname members users whoami who w
WireGuard VPNAlpine CentOS 8 Debian 10 Firewall Ubuntu 20.04
5 comments… add one
  • Raj dixit Jul 25, 2017 @ 14:38

    There are many instances when you have mentioned it as ipref instead of ipref.

  • nc fan Jul 25, 2017 @ 16:42

    How about just using nc? On the server:
    nc -vvlnp 5203 >/dev/null
    On the client:
    dd if=/dev/zero bs=1M count=1K | nc -vvn 192.168.149.69 5203
    I see
    Connection to 192.168.149.69 5203 port [tcp/*] succeeded!
    1024+0 records in
    1024+0 records out
    1073741824 bytes (1.1 GB, 1.0 GiB) copied, 4.6777 s, 230 MB/s

    Easy and no need to install iperf tool.

  • David Aug 1, 2017 @ 11:00

    A non-intrusive way of measuring the bandwidth of the link connecting two servers, on symmetric links, is using pings of different sizes, e.g. 64 and 1064 bytes, and measuring the RTT, then divide the difference of ping sizes in bits by the difference of measured RTTs and you get the link bandwidth.

  • Melroy van den Berg Nov 8, 2020 @ 2:20

    UDP test doesn’t require the -u option on the server side (ServerA), only the client using -u is sufficient enough.

Leave a Reply

Your email address will not be published.

Use HTML <pre>...</pre> for code samples. Still have questions? Post it on our forum