nixCraft

How to find disk I/O latency with ioping monitoring tool on Linux

How do I monitor disk I/O latency in real time on Linux, FreeBSD, OpenBSD, and MacOS operating system?

You can use ioping tool to monitor I/O latency in real time. It shows disk latency in the same way as ping command shows network latency on Linux or Unix-like system.

Installation

Install as per your operating system.

Install ioping on a Debian/Ubuntu/Mint Linux

Type the following apt-get command/apt command:
$ sudo apt-get install ioping

Install ioping on an Arch Linux

Type the following pacman command:
$ sudo pacman -S ioping

Install ioping on a Fedora Linux

Type the following dnf command:
$ sudo dnf install ioping

Install ioping on a macOS

Type the following brew command:
$ brew install ioping
Sample outputs:

==> Downloading https://homebrew.bintray.com/bottles/ioping-1.0.sierra.bottle.tar.gz
######################################################################## 100.0%
==> Pouring ioping-1.0.sierra.bottle.tar.gz
*  /usr/local/Cellar/ioping/1.0: 7 files, 79.4KB

Install ioping on a FreeBSD

Type the following pkg command:
# pkg install ioping
Sample outputs:

Updating FreeBSD repository catalogue...
Fetching meta.txz: 100%    944 B   0.9kB/s    00:01    
Fetching packagesite.txz: 100%    6 MiB   6.0MB/s    00:01    
Processing entries: 100%
FreeBSD repository update completed. 26484 packages processed.
All repositories are up to date.
The following 1 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
	ioping: 1.0

Number of packages to be installed: 1

23 KiB to be downloaded.

Proceed with this action? [y/N]: y
[1/1] Fetching ioping-1.0.txz: 100%   23 KiB  23.6kB/s    00:01    
Checking integrity... done (0 conflicting)
[1/1] Installing ioping-1.0...
Extracting ioping-1.0: 100%

Install ioping on a OpenBSD

Type the following pkg_add command:
# pkg_add ioping
Sample outputs:

quirks-2.304 signed on 2017-04-02T15:01:33Z
ioping-1.0: ok

How do I use ioping?

Warning: Use the ioping command with care. Wrong options can destroy your data. Read man page for more info.

The syntax is:
ioping /tmp/
ioping /dev/sda1
ioping [options] device/dir/file

See disk I/O latency using the default values and the current directory

$ ioping .
To stop after 12 count, pass the -c option:
$ ioping -c 12 .
Sample outputs from OpenBSD running on Linode server:

ioping: non-cached I/O not supported by this platform
ioping: you can use write I/O to get reliable results
4 KiB 

To see disk seek rate

$ sudo ioping -R /dev/sda

To see disk sequential speed (MiB/s)

$ sudo ioping -RL /dev/rdisk1
Sample outputs:

Fig.01: Finding disk seek rate and disk sequential speed

How to set interval between requests

The default is 1 second. To change pass the -i {interval} option. Set 3 seconds:
$ ioping -i 3 .
$ ioping -i 3 /dev/sdb

How do I use asynchronous I/O?

Pass the -A option:
# ioping -c 6 -A /dev/vda1
Sample outputs:

4 KiB from /dev/vda1 (block device 20.0 GiB): request=1 time=458 us
4 KiB from /dev/vda1 (block device 20.0 GiB): request=2 time=226 us
4 KiB from /dev/vda1 (block device 20.0 GiB): request=3 time=477 us
4 KiB from /dev/vda1 (block device 20.0 GiB): request=4 time=412 us
4 KiB from /dev/vda1 (block device 20.0 GiB): request=5 time=381 us
4 KiB from /dev/vda1 (block device 20.0 GiB): request=6 time=356 us

--- /dev/vda1 (block device 20.0 GiB) ioping statistics ---
6 requests completed in 5.00 s, 2.60 k iops, 10.1 MiB/s
min/avg/max/mdev = 226 us / 385 us / 477 us / 82 us

How to use cached I/O (no cache flush/drop)

Pass the -C option:
$ ioping -c 6 -C /dev/sdb1
# ioping -c 6 -C .

Sample outputs from FreeBSD server hosted at Linode:

4 KiB 

How to use direct I/O (O_DIRECT)

Pass the -D option:
# ioping -c 6 -D /tmp
Sample outputs from FreeBSD+ZFS based fs:

4 KiB 

How to use write I/O (please read manpage)

Pass the -W option. From the man page:

Safe for directory target. Write I/O gives more reliable results for systems where non-cached reads are not supported or cached at some level. Might be *DANGEROUS* for file/device: it will shred your data. In this case should be repeated tree times (-WWW).

$ ioping -c 6 -W /tmp
Sample outputs from OpenBSD sever at Linode:

4 KiB >>> /tmp (ffs /dev/sd0d): request=1 time=363.9 us (warmup)
4 KiB >>> /tmp (ffs /dev/sd0d): request=2 time=684.4 us
4 KiB >>> /tmp (ffs /dev/sd0d): request=3 time=6.07 ms
4 KiB >>> /tmp (ffs /dev/sd0d): request=4 time=2.85 ms
4 KiB >>> /tmp (ffs /dev/sd0d): request=5 time=7.98 ms
4 KiB >>> /tmp (ffs /dev/sd0d): request=6 time=1.33 ms

--- /tmp (ffs /dev/sd0d) ioping statistics ---
5 requests completed in 18.9 ms, 20 KiB written, 264 iops, 1.03 MiB/s
generated 6 requests in 5.01 s, 24 KiB, 1 iops, 4.79 KiB/s
min/avg/max/mdev = 684.4 us / 3.78 ms / 7.98 ms / 2.80 ms

How to use read-write ping-pong mode

Pass the -G option:
# ioping -c 10 -G /tmp
Sample outputs Google cloud VM:

4 KiB >>> /tmp (ext4 /dev/sda1): request=1 time=614.4 us (warmup)
4 KiB >> /tmp (ext4 /dev/sda1): request=3 time=1.05 ms
4 KiB >> /tmp (ext4 /dev/sda1): request=5 time=965.0 us
4 KiB >> /tmp (ext4 /dev/sda1): request=7 time=1.17 ms
4 KiB >> /tmp (ext4 /dev/sda1): request=9 time=1.14 ms
4 KiB 

How to use use sync I/O (O_SYNC)

Pass the -Y option:
# ioping -c 10 -Y /dev/sdb
Sample outputs from Google compute VM:

4 KiB 

How to use use data sync I/O (O_DSYNC)

Pass the -y option:
$ ioping -c 10 -y .
Sample outputs from macos:

Fig.02: Use data sync I/O (see O_DSYNC in open(2)).

Getting help

Read man page:
$ man ioping
OR
$ ioping -h
Sample outputs:

      -c       stop after  requests
      -i    interval between requests (1s)
      -l       speed limit in bytes per second
      -t 

See ioping home page for more info.

Get 15% off on Linux Foundation certified SysAdmin, Progamming, Kubernetes/Containers and Open Stack certification & course. Use "SPLASH15" coupon code. Offer expires on August 27, 2018 training.linuxfoundation.org