How do I find out Linux Disk utilization?

by nixCraft on December 17, 2007 · 17 comments· Last updated December 17, 2007

I've already written about finding out Linux / UNIX cpu utilization using various tools. You can use same iostat command to find out disk utilization and for monitoring system input/output device loading by observing the time the physical disks are active in relation to their average transfer rates.

iostat syntax for disk utilization report

iostat -d -x interval count

-d : Display the device utilization report (d == disk)
-x : Display extended statistics including disk utilization
interval : It is time period in seconds between two samples . iostat 2 will give data at each 2 seconds interval.
count : It is the number of times the data is needed . iostat 2 5 will give data at 2 seconds interval 5 times

Display 3 reports of extended statistics at 5 second intervals for disk

Type the following command:
$ iostat -d -x 5 3
Output:

Linux 2.6.18-53.1.4.el5 (moon.nixcraft.in)   12/17/2007
Device:         rrqm/s   wrqm/s   r/s   w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               1.10    39.82  3.41 13.59   309.50   427.48    43.36     0.17   10.03   1.03   1.75
sdb               0.20    18.32  1.15  6.08   117.36   195.25    43.22     0.51   71.14   1.26   0.91
Device:         rrqm/s   wrqm/s   r/s   w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               0.00   108.40  1.40 64.40    49.60  1382.40    21.76     0.04    0.67   0.44   2.92
sdb               0.00    37.80  0.00 245.20     0.00  2254.40     9.19    28.91  108.49   1.08  26.36
Device:         rrqm/s   wrqm/s   r/s   w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               0.00    97.01  1.00 57.29    39.92  1234.33    21.86     0.03    0.58   0.50   2.89
sdb               0.00    38.32  0.00 288.42     0.00  2623.55     9.10    32.97  122.30   1.15  33.27

Where,

rrqm/s : The number of read requests merged per second that were queued to the hard disk
wrqm/s : The number of write requests merged per second that were queued to the hard disk
r/s : The number of read requests per second
w/s : The number of write requests per second
rsec/s : The number of sectors read from the hard disk per second
wsec/s : The number of sectors written to the hard disk per second
avgrq-sz : The average size (in sectors) of the requests that were issued to the device.
avgqu-sz : The average queue length of the requests that were issued to the device
await : The average time (in milliseconds) for I/O requests issued to the device to be served. This includes the time spent by the requests in queue and the time spent servicing them.
svctm : The average service time (in milliseconds) for I/O requests that were issued to the device
%util : Percentage of CPU time during which I/O requests were issued to the device (bandwidth utilization for the device). Device saturation occurs when this value is close to 100%.

How do I interpret the output result for optimization?

First you need to note down following values from the iostat output:

The average service time (svctm)
Percentage of CPU time during which I/O requests were issued (%util)
See if a hard disk reports consistently high reads/writes (r/s and w/s)

If any one of these are high, you need to take one of the following action:

Get high speed disk and controller for file system (for example move from SATA I to SAS 15k disk)
Tune software or application or kernel or file system for better disk utilization
Use RAID array to spread the file system

For example, from about iostat report it appears that /dev/sdb under load. Hope this information will help you diagnose and optimize disk related issues.

Related: How to find out Linux CPU utilization using vmstat, iostat, mpstat and sar commands.

Please note that command and information discussed here almost applies to any other UNIX like variant.

You should follow me on twitter here or grab rss feed to keep track of new changes.

Featured Articles:

{ 17 comments… read them below or add one }

1 Rishi kapur February 29, 2008 at 5:54 am

Please suggest me – system load average is shooting.

%iowait is high and found with iostat output one disk mentioned below may be the cause.

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
cciss/c0d0 2269.60 132.60 296.20 77.60 20521.60 1681.60 10260.80 840.80 59.40 4.15 11.18 2.65 99.20

please suggest

2 Mike March 1, 2008 at 1:23 pm

When I enter the command “iostat -d -x 5 3″ I do not get any statistics. Here is what I get:

# iostat -d -x 5 3
Linux 2.4.33.3 (FILES) 03/01/2008

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util

Any ideas on what the problem could be? I’m running Slackware 11.0 with the stock kernel.

Thanks in advance.

3 soundar February 9, 2012 at 12:55 pm: where x = device
example : iostat -x -d sda 5
Reply

4 Ash Sethi May 21, 2008 at 6:31 am

Another good article… thanks for sharing,

5 whall August 27, 2009 at 9:43 pm

Mike – you need to not specify “-d”. Only use -d if you’re going to specify a disk like sda or sdb. Just use the -x interval.

ie, iostat -x 10

means it will immediately give one output that represents from boot to now, and then every 10 seconds it will output a line that represents the last 10 seconds, and it will repeat forever.

You can also take this into excel and graph it, paying attention to the 4-5 columns the author mentioned.

6 arumon September 3, 2009 at 11:48 am

good artice!!!!
one question, Please tell what is the use of the below specified files.
/sys/block/sda/queue/iosched/queued
/sys/block/sda/queue/nr_requests
/sys/block/sda/device/queue_depth

7 Dave Neary February 9, 2010 at 11:28 am

Hi!

Thanks for the information, very useful.

I have one question – what constitutes “high” for r/s & w/s? Is over 10 high? Over 100, 1000?

I suspect that a hard disk is slowing down on me – whenever there is an I/O intensive operation (swapping, download, starting Firefox or OOo, things like that), everything slows to a crawl, including process switching.

Is there a way to find out from iostat what bandwidth the disk is giving me? What is the real latency of a disk?

Thanks!
Dave.

8 David Ward August 16, 2010 at 5:03 am

Great article. Thanks!

I too would like to know what is “high”

Also, how do you follow on from this. I have had massive IO wait issues at times and tracking down the process causing the issue is very hard. Any tips in this direction?

Thanks

9 David Ward August 16, 2010 at 5:03 am

I should use my gravatar email address :)

10 Lars July 26, 2011 at 9:05 am

Great article. Just a quick note. On one of our older Debian servers we had to install sysstats package to use iostat.

11 Joseph Taiwo August 9, 2011 at 6:08 am

Hi,

My company uses a VPS hosted in softlayer. Anytime Irun iostat, I hadly get any of these data. Could it be a problem with the VPS?
The sample of the command output for iostat -d -x 5 3 is as below:

Device:         rrqm/s   wrqm/s   r/s   w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
Device:         rrqm/s   wrqm/s   r/s   w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
Device:         rrqm/s   wrqm/s   r/s   w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util

Please I will like to know if there is a way around this. Thanks

12 Reza September 2, 2011 at 5:03 pm

Very useful… thanks!

13 amir April 24, 2012 at 11:00 am

iostat man page :
The average service time (svctm field) value is meaningless, as I/O statistics are calculated at block level, and we don’t know when the disk driver starts to process a request. For this reason, this field will be removed in a future sysstat version.

14 Suchitra Behera October 26, 2012 at 8:29 am

Please let me know ..how to identify what is causing high disk utilization ??

15 Raj February 27, 2013 at 9:16 pm

Hey What are the statistics for high – Just information. Need to be more specific.
If you can not evaluate the information and reach to the conclusion then its of no use.

Thanks

16 Martin April 9, 2013 at 8:36 pm

Here’s how you get the information filtered

iostat -x 5 3 | sed "s: \+: :g" | cut -d ' ' -f 1,4,5,13,14  | sed "s: :\t:g"

you would have to wait the full 15 seconds before you see any data.
So this is an even nicer way.

for n in {1..3}; do iostat -x 1 1 | sed "s: \+: :g" | cut -d ' ' -f 1,4,5,13,14  | sed "s: :\t:g" | tail -n +2 && sleep 5; done

17 ravi April 30, 2013 at 7:27 am

Hi,

when i try to capture the disk i/o using iostat in linux server i was getting the below error.
iostat
If ‘iostat’ is not a typo you can run the following command to lookup the package that contains the binary:
command-not-found iostat
bash: iostat: command not found

please tel me the alternative command for iostat in Linux.
i have tried iotop,it was not working.

Brs,
Ravi