nixCraft Poll

Topics

How do I find out Linux CPU utilization?

Posted by Vivek Gite [Last updated: December 5, 2007]

Whenever a Linux system CPU is occupied by a process, it is unavailable for processing other requests. Rest of pending requests must wait till CPU is free. This becomes a bottleneck in the system. Following command will help you to identify CPU utilization, so that you can troubleshoot CPU related performance problems.

Finding CPU utilization is one of the important tasks. Linux comes with various utilities to report CPU utilization. With these commands, you will be able to find out:

* CPU utilization
* Display the utilization of each CPU individually (SMP cpu)
* Find out your system's average CPU utilization since the last reboot etc
* Determine which process is eating the CPU(s)

Old good top command

The top program provides a dynamic real-time view of a running system. It can display system summary information as well as a list of tasks currently being managed by the Linux kernel.
The top command monitors CPU utilization, process statistics, and memory utilization. The top section contains information related to overall system status - uptime, load average, process counts, CPU status, and utilization statistics for both memory and swap space.

Top command

Type the top command:
$ top

Output:

You can see Linux CPU utilization under CPU stats. The task’s share of the elapsed CPU time since the last screen update, expressed as a percentage of total CPU time. In a true SMP environment (multiple CPUS), top will operate in number of CPUs. Please note that you need to type q key to exit the top command display.

Find Linux CPU utilization using mpstat and other tools

Please note that you need to install special package called sysstat to take advantage of following commands. This package includes system performance tools for Linux (Red Hat Linux / RHEL includes these tools by default).

# apt-get install sysstat
Use up2date command if you are using RHEL:
# up2date sysstat

Display the utilization of each CPU individually using mpstat

If you are using SMP (Multiple CPU) system, use mpstat command to display the utilization of each CPU individually. It report processors related statistics. For example, type command:
# mpstat Output:

Linux 2.6.15.4 (debian)         Thursday 06 April 2006

05:13:05  IST  CPU   %user   %nice    %sys %iowait    %irq   %soft  %steal   %idle    intr/s
05:13:05  IST  all   16.52    0.00    2.87    1.09    0.07    0.02    0.00   79.42    830.06

The mpstat command display activities for each available processor, processor 0 being the first one. Global average activities among all processors are also reported. The mpstat command can be used both on SMP and UP machines, but in the latter, only global average activities will be printed.:
# mpstat -P ALL
Output:

Linux 2.6.15.4 (wwwportal1.xxxx.co.in)         Thursday 06 April 2006

05:14:58  IST  CPU   %user   %nice    %sys %iowait    %irq   %soft  %steal   %idle    intr/s
05:14:58  IST  all   16.46    0.00    2.88    1.08    0.07    0.02    0.00   79.48    835.96
05:14:58  IST    0   16.46    0.00    2.88    1.08    0.07    0.02    0.00   79.48    835.96
05:14:58  IST    1   15.77    2.70    3.17    2.01    0.05    0.03    0.00   81.44    822.54

Another output from my HP Dual Opteron 64 bit server:# mpstat -P ALLOutput:

Linux 2.6.5-7.252-smp (ora9.xxx.in)   04/07/06

07:44:18     CPU   %user   %nice %system %iowait    %irq   %soft   %idle    intr/s
07:44:18     all    3.01   57.31    0.36    0.13    0.01    0.00   39.19   1063.46
07:44:18       0    5.87   69.47    0.44    0.05    0.01    0.01   24.16    262.11
07:44:18       1    1.79   48.59    0.36    0.23    0.00    0.00   49.02    268.92
07:44:18       2    2.19   42.63    0.28    0.16    0.01    0.00   54.73    260.96
07:44:18       3    2.17   68.56    0.34    0.06    0.03    0.00   28.83    271.47

Report CPU utilization using sar command

You can display today’s CPU activity, with sar command:
# sar
Output:

Linux 2.6.9-42.0.3.ELsmp (dellbox.xyz.co.in)         01/13/2007

12:00:02 AM       CPU     %user     %nice   %system   %iowait     %idle
12:10:01 AM       all      1.05      0.00      0.28      0.04     98.64
12:20:01 AM       all      0.74      0.00      0.34      0.38     98.54
12:30:02 AM       all      1.09      0.00      0.28      0.10     98.53
12:40:01 AM       all      0.76      0.00      0.21      0.03     99.00
12:50:01 AM       all      1.25      0.00      0.32      0.03     98.40
01:00:01 AM       all      0.80      0.00      0.24      0.03     98.92
...
.....
..
04:40:01 AM       all      8.39      0.00     33.17      0.06     58.38
04:50:01 AM       all      8.68      0.00     37.51      0.04     53.78
05:00:01 AM       all      7.10      0.00     30.48      0.04     62.39
05:10:01 AM       all      8.78      0.00     37.74      0.03     53.44
05:20:02 AM       all      8.30      0.00     35.45      0.06     56.18
Average:          all      3.09      0.00      9.14      0.09     87.68

Comparison of CPU utilization

The sar command writes to standard output the contents of selected cumulative activity counters in the operating system. The accounting system, based on the values in the count and interval parameters. For example display comparison of CPU utilization; 2 seconds apart; 5 times, use:
# sar -u 2 5
Output (for each 2 seconds. 5 lines are displayed):

Linux 2.6.9-42.0.3.ELsmp (www1lab2.xyz.ac.in)         01/13/2007

05:33:24 AM       CPU     %user     %nice   %system   %iowait     %idle
05:33:26 AM       all      9.50      0.00     49.00      0.00     41.50
05:33:28 AM       all     16.79      0.00     74.69      0.00      8.52
05:33:30 AM       all     17.21      0.00     80.30      0.00      2.49
05:33:32 AM       all     16.75      0.00     81.00      0.00      2.25
05:33:34 AM       all     14.29      0.00     72.43      0.00     13.28
Average:          all     14.91      0.00     71.49      0.00     13.61

Where,

To get multiple samples and multiple reports set an output file for the sar command. Run the sar command as a background process using.
# sar -o output.file 12 8 >/dev/null 2>&1 &
Better use nohup command so that you can logout and check back report later on:
# nohup sar -o output.file 12 8 >/dev/null 2>&1 &

All data is captured in binary form and saved to a file (data.file). The data can then be selectively displayed ith the sar command using the -f option.
# sar -f data.file

Task: Find out who is monopolizing or eating the CPUs

Finally, you need to determine which process is monopolizing or eating the CPUs. Following command will displays the top 10 CPU users on the Linux system.
# ps -eo pcpu,pid,user,args | sort -k 1 -r | head -10
OR
# ps -eo pcpu,pid,user,args | sort -r -k1 | less
Output:

%CPU   PID USER     COMMAND
  96  2148 vivek    /usr/lib/vmware/bin/vmware-vmx -C /var/lib/vmware/Virtual Machines/Ubuntu 64-bit/Ubuntu 64-bit.vmx -@ ""
 0.7  3358 mysql    /usr/libexec/mysqld --defaults-file=/etc/my.cnf --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --socket=/var/lib/mysql/mysql.sock
 0.4 29129 lighttpd /usr/bin/php
 0.4 29128 lighttpd /usr/bin/php
 0.4 29127 lighttpd /usr/bin/php
 0.4 29126 lighttpd /usr/bin/php
 0.2  2177 vivek    [vmware-rtc]
 0.0     9 root     [kacpid]
 0.0     8 root     [khelper]

Now you know vmware-vmx process is eating up lots of CPU power. ps command displays every process (-e) with a user-defined format (-o pcpu). First field is pcpu (cpu utilization). It is sorted in reverse order to display top 10 CPU eating process.

iostat command

You can also use iostat command which report Central Processing Unit (CPU) statistics and input/output statistics for devices and partitions. It can be use to find out your system's average CPU utilization since the last reboot.
# iostatOutput:

Linux 2.6.15.4 (debian)         Thursday 06 April 2006

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
     16.36    0.00    2.99    1.06    0.00   79.59

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
hda               0.00         0.00         0.00         16          0
hdb               6.43        85.57       166.74     875340    1705664
hdc               0.03         0.16         0.00       1644          0
sda               0.00         0.00         0.00         24          0 

You may want to use following command, which gives you three outputs every 5 seconds (as previous command gives information since the last reboot):$ iostat -xtc 5 3

GUI tools for your laptops/desktops

Above tools/commands are quite useful on remote server. For local system with X GUI installed you can try out gnome-system-monitor. It allows you to view and control the processes running on your system. You can access detailed memory maps, send signals, and terminate the processes.
$ gnome-system-monitor

gnome-system-monitor - view and control the processes
(Click to enlarge image)

In addition, the gnome-system-monitor provides an overall view of the resource usage on your system, including memory and CPU allocation.

gnome-system-monitor - view and control the processes
(Click to enlarge image)

Further readings

Want to stay up to date with the latest Linux tips, news and announcements? Subscribe to our free e-mail newsletter or RSS feed to get all updates. You can Email this page to a friend.

You may also be interested in other helpful articles:

Discussion on This Article:

  1. Yuchi Says:

    Sounds interesting

  2. Anonymous Says:

    Never knew the mpstat command. Thanks for post. You got kick ass info :D

  3. Anonymous Says:

    So you have 4 processors shown on your dual opteron machine which means that 2 of them are virtual. How do youfilter these out when you want to see only raw info on real processor utilization?

    Thanks

    Nikola

  4. Anonymous Says:

    I need to explain better i think:
    so i have 4 processors and what if only 2 are working and i still see 4 because of additional virtual processors? How do i check that? Plz answer its really important to me.

    Thanks in advance,

    Nikola

  5. nixcraft Says:

    If cpu entry not present in /proc/interrupts file your cpu is offline or not working for some causes:
    less /proc/interrupts

    You can also goto directory /sys/devices/system/cpu (note following commands needs special configuration option via kernel. If it is not complied it will not work for you):
    cd /sys/devices/system/cpu

    Type ls command to see all cpus
    ls

    Output:
    cpu0 cpu1 cpu2 cpu3 cpu4 cpu5 cpu6 cpu7

    Inside this directory you will see entry for online or offline CPU

    Hope this helps

  6. Ashok Says:

    Thanks for the commands - i really could use the iostat command and get some answers.. Thanks Again !! :)

  7. nixcraft Says:

    Ashok,

    I am glad - commands and help presented here helping out.

    Appreciate your post.

  8. chadi Says:

    hi all i have a prob i use top and i have mysql has a %cpu 99.5 and i use mpstat but i have not the same results why with us and sy and ni
    i have smp linux rehl3

  9. nixcraft Says:

    MySQL is generally use more disk i/o; use iostat to get detailed information. You need to optimize mysql and also get fast SCSI hard disk.

  10. Planet Malaysia Says:

    Yeah! Now I know new command “mpstat”

  11. Pádraig Brady Says:

    This is the command I use to see who’s using CPU:

    ps -e -o pcpu,cpu,nice,state,cputime,args –sort pcpu | sed ‘/^ 0.0 /d’

    That and more here:
    http://www.pixelbeat.org/cmdline.html#monitor

  12. nixcraft Says:

    Pádraig,

    Good syntax, just to list top 10, sorting should be -pcpu and pipe to head -10

    ps -e -o pcpu,cpu,nice,state,cputime,args –sort -pcpu | sed ‘/^ 0.0 /d’ | head -10

    Appreciate your post.

  13. Markus Sorensson Says:

    Ok, I can measure the cpu utilization… but what is the criteria to determine when a cpu is overloaded? Please let me know!

  14. SpongeMucker Says:

    Good Article. However, you neglected to mention vmstat. One of the nice things about vmstat is that it provides an insight into how the queues are filling up on each processor. Also, vmstat comes with just about every default install of UNIX and Linux that I have ever seen, so there are no additional files which need to be installed. vmstat is also very scriptable, if you need to log the cpu usage at intervals of time. Hope that this helps. Thanks again for a good article.

  15. jon Says:

    Just FYI - on fedora machines, you’ll need to:

    $ yum install sysstat

    To install the sysstat package which contains (most of) the binaries listed in this article.

  16. nixcraft Says:

    @Markus when you see CPU load >=70%

    @SpongeMucker: yup vmstat is good tool => http://www.cyberciti.biz/tips/linux-resource-utilization-to-detect-system-bottlenecks.html. I will update post with vmstat link.

    @ jon : thanks for pointing out fedora core issue.

    Appreciate all of your posts.

  17. JM Says:

    htop works great for me. It’s a little easier on the eyes than regular old top.

  18. Santiago Says:

    You can use the “atop” utility(Monitor for system resources and process activity)

  19. nixcraft Says:

    Santiago/jm

    Yup, atop/htop is also useful.

    Appreciate your posts.

  20. drown Says:

    yes its very good explaination..
    but how can one get this information like cpu usage into a program.. say C code?..
    say i wanted cpu usage into my float variable..
    please help.. thnks in advance…

  21. nixcraft Says:

    Drown,

    I am not sure about C API.

  22. blackice Says:

    @19. nixcraft

    Thanks, htop worked great for debian sarge, mpstat didn’t worked very well.

  23. Puneet Says:

    Thanks man…helped a lot..:)

    ~Puneet

  24. Vishal Says:

    Thanks a lot it is really helpful.!!

  25. Diptanjan Says:

    This is really a wonderful article…

    Learned a lot from this… And thank you all the readers for posting valuable comments with different commands.. this is really helpful for people like me.

    Diptanjan

  26. Ramanath Says:

    This is a nice and informative article.
    Can anyone suggest any command to monitor a process is using which cpu in SMP environment. I mean suppose I am running sshd. How do I know it is using cpu0 or cpu1 ?

  27. benny Says:

    Thanks a lot for the post. Its really helpful.

  28. MG Says:

    very good information…it would be good if we can have at least some hints on whether the stats shown is still ok or need to check on other items

  29. colbert Says:

    thanks for the tips. its good read for my day to day job

  30. invisible Says:

    Really good article.

    I have a following question - what is the best way to find out which process(es) was(were) using cpu(s) for certain period of time based on cpu utilization threshold?

    Let’s imagine that we want to find all processes triggering CPU utilization beyond 70% for more then 10 seconds during last 24 hours and dump such an information in a log with the snapshots of time of occurrence, process ID and command.

    Anyone has an idea how to setup such a monitoring in an elegant way?

    Thanks,
    –invisible

  31. laga Says:

    Thanks for the mpstat command!

  32. zhili wang Says:

    One of our server has four CPUs, but when I use mpstat or top, I can only see one CPU. In the directory /sys/devices/system/cpu, only show cpu0. But we do have 4 cpu, and OS admin told me all four cpu are runing. How do I know all four cpus are runing properly and I can use it.

    Thanks for any help.

  33. vivek Says:

    zhili,

    Install SMP Linux kernel RPM / package and reboot the box. Run top or cat /proc/cpuinfo command to confirm CPU count

  34. zhili wang Says:

    my server has four CPUs, but when I issue vmstat and top, I can only see one cpu. In the directory:/sys/devices/system/cpu, I can see only one cpu - cpu0. When I issue command:less /proc/interrupts, it only show one cpu. But there are do have four cpu on the server, should I do something to configure it, what should I do.

    Thanks for any help.

  35. zhili wang Says:

    Vivek,

    Thank you for your help, I will let Linux Admin here do it.

  36. vivek Says:

    >should I do something to configure it, what should I do

    Yup, ask your admin to install Linux SMP kernel and boot into the same using Grub. Once booted using SMP, run uname -a to verify that SMP kernel loaded. Also go through /proc/cpuinfo file.
    less /proc/cpuinfo

  37. zhili wang Says:

    The Linux Admin told me kernel-smp are package for free linux and some old version, we runing advanced linux version and do not need it, it confused me. Is it true?

  38. zhili wang Says:

    Thanks for all the help.

    It is clear now, we did not install smp linux kernel, so there is only one cpu working. Could any one tell me when install Redhat linux on multip cpu server, it will automatically configure to smp kernel or have to maually select the kernel?

  39. vivek Says:

    zhili,

    In most cases it is installed by default but sometime installer cannot detect it and it will install normal kernel. Run yum / up2date command to install SMP kernel from RHN.

  40. zhili wang Says:

    Vivek,

    It is very helpful. Thank you.

  41. olfat Says:

    i find this howto article very useful. thank you!

  42. ben Says:

    is there a GUI version of a CPU usage monitor for KDE? im using fedora 8

  43. Daton Says:

    Hi, I have a question about iostat command. How can I see the %utilization of disk not %utilization of CPU? I have found out from some of the forums and man page for iostat that the disk utilization report ought to show one parameter called %util. And also somewhere I have found a flag -D to show it but I cannot find anywhere how to configure my command so that it shows this %util field.

    Any help is highly appreciated. Thank you.

  44. vivek Says:

    Daton,

    Sometime man page can be confusing..just run iostat -d -x to display disk utilization including %util. It will show TPS (number of transfers per second ) and amount of data read and/or wrtite to/from the device. Try following examples,
    iostat -d -x
    iostat -d -m -x
    iostat -x -d 2

    All you have to do is pass -x option.

  45. Madivanan Says:

    everything in a nutshell..excellent Good info

  46. Siddharth Says:

    Very useful information and very well presented! Thanks!

  47. uncleremus Says:

    You forgot the nicest monitoring tool of them all: gkrellm. Small, fast, nice-looking, must have. Check it out.

  48. Vitaly Says:

    when we see that certain process takes 10% of CPU and we have 2 CPU server, does it mean - it is 10% from one CPU (==5% of server CPU power ) or 10% from server CPU power (==20% of one CPU)

  49. Jamie Says:

    HI nice tute!

    Same question as Vitaly above.

    Ina 2 cpus enviroenmnt does the cpu usage percentage mean the percentage the process takes of 2 cpu or both. Please see the above exmple by Vitaly.

    Also in your image of the top output. The cpu usage vales for the processes add up to a value (26.8%) greater than the total cpu usage value 23.%)

  50. Jacob Says:

    You forgot ksysguard in your GUI section. It has a LOT more monitoring ability than the gnome-system-monitor. Here’s a nice article covering its pros and cons:
    http://www.linux.com/articles/113700

  51. Pirkia.lt admin Says:

    top command is better use with argument c (you will get more information):

    top c

  52. Linux In Israel Says:

    Great article! sar is not so popular as it should be …

  53. Nicolas Says:

    Great intro to CPU usage. Thanks.

    The top 10 monopolizing process would be better if sorted as a numberic key :


    ps -eo pcpu,pid,user,args | sort -k 1 -r -n | head -10

  54. Michael Says:

    I have a dual core system. When I type ‘mpstat -P ALL’

    10:52:18 PM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
    10:52:18 PM all 14.33 0.13 2.49 0.43 0.03 0.18 0.00 82.41 151.04
    10:52:18 PM 0 15.47 0.13 2.47 0.33 0.00 0.01 0.00 81.59 0.00
    10:52:18 PM 1 13.21 0.13 2.51 0.53 0.07 0.35 0.00 83.21 151.04
    10:52:18 PM 2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

    What is CPU 2?

  55. Satya Says:

    Hi All,

    Today is 5/28/2008 but I ran some performance tests during 5/7/2008.I need to capture the measurements for both Memory and CPU using ‘SAR’ commands for each of 1 minute interval. Is it possible? If yes, what commands should I need to use?

    I appreciate your input here.

    Thanks
    Satya

  56. Giuseppe Says:

    What do you think about cyclesoak?
    I wonder if you could provide some example by using it as well.

  57. s5h Says:

    Thanks for writing this, it’s a very good document about how to use proc. Excellent work. Things like this should be in the kernel docs but they just aren’t. Thanks for filling in the gaps.

Leave a Reply

We encourage your comments, and suggestions. But please stay on topic, be polite, and avoid spam. Thank you very much for stopping by our site!

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

*
To prove you're a person (not a spam script), type the security word shown in the picture. Click on the picture to hear an audio file of the word.
Click to hear an audio file of the anti-spam word

Tags: , , , , , , , , , , , , , , , , , , , ,

Copyright © 2004-2008 nixCraft. All rights reserved - TOS/Disclaimer - Privacy policy - Sitemap - Powered by Open source software.