How do I Find Out Linux CPU Utilization?

Posted on in Categories Linux, Sys admin, Tips, Troubleshooting, Tuning last updated December 16, 2015

Whenever a Linux system CPU is occupied by a process, it is unavailable for processing other requests. Rest of pending requests must wait till CPU is free. This becomes a bottleneck in the system. Following command will help you to identify CPU utilization, so that you can troubleshoot CPU related performance problems on a Linux based system.

Finding CPU utilization is one of the important tasks. Linux comes with various utilities to report CPU utilization. With these commands, you will be able to find out:

  1. CPU utilization
  2. Display the utilization of each CPU individually (SMP cpu)
  3. Find out your system’s average CPU utilization since the last system reboot
  4. Determine which process is eating the CPU(s)

The old good top command to find out Linux cpu load

The top program provides a dynamic real-time view of a running system. It can display system summary information as well as a list of tasks currently being managed by the Linux kernel. The top command monitors CPU utilization, process statistics, and memory utilization. The top section contains information related to overall system status – uptime, load average, process counts, CPU status, and utilization statistics for both memory and swap space.

Top command to find out Linux cpu usage

Type the top command:
$ top
Sample outputs:

top command in action on my Linux box
Fig.01: top command in action (click to enlarge)

You can see Linux CPU utilization under CPU stats. The task’s share of the elapsed CPU time since the last screen update, expressed as a percentage of total CPU time. In a true SMP environment (multiple CPUS), top will operate in number of CPUs. Please note that you need to type q key to exit the top command display.

The top command produces a frequently-updated list of processes. By default, the processes are ordered by percentage of CPU usage, with only the “top” CPU consumers shown. The top command shows how much processing power and memory are being used, as well as other information about the running processes.

Find Linux CPU utilization using mpstat and other tools

Please note that you need to install a special package called sysstat to take advantage of following commands. This package includes system performance tools for Linux (Red Hat Linux / RHEL includes these tools by default). Install it on a Debian or Ubuntu Linux using apt-get command:
# apt-get install sysstat
Use up2date command if you are using RHEL/CentOS Linux v4.x or older:
# up2date install sysstat
Use up2date command if you are using RHEL/CentOS Linux v5.x+ or newer:
# yum install sysstat

Display the utilization of each CPU individually using mpstat

If you are using SMP (Multiple CPU) system, use mpstat command to display the utilization of each CPU individually. It report processors related statistics. For example, type command:
# mpstat
Sample outputs:

Linux 2.6.15.4 (debian)         Thursday 06 April 2006
 
05:13:05  IST  CPU   %user   %nice    %sys %iowait    %irq   %soft  %steal   %idle    intr/s
05:13:05  IST  all   16.52    0.00    2.87    1.09    0.07    0.02    0.00   79.42    830.06

\
The mpstat command display activities for each available processor, processor 0 being the first one. Global average activities among all processors are also reported. The mpstat command can be used both on SMP and UP machines, but in the latter, only global average activities will be printed.:
# mpstat -P ALL
Sample outputs:

Linux 2.6.15.4 (wwwportal1.xxxx.co.in)         Thursday 06 April 2006
 
05:14:58  IST  CPU   %user   %nice    %sys %iowait    %irq   %soft  %steal   %idle    intr/s
05:14:58  IST  all   16.46    0.00    2.88    1.08    0.07    0.02    0.00   79.48    835.96
05:14:58  IST    0   16.46    0.00    2.88    1.08    0.07    0.02    0.00   79.48    835.96
05:14:58  IST    1   15.77    2.70    3.17    2.01    0.05    0.03    0.00   81.44    822.54

Another output from my HP Dual Opteron 64 bit server:# mpstat -P ALLOutput:

Linux 2.6.5-7.252-smp (ora9.xxx.in)   04/07/06

07:44:18     CPU   %user   %nice %system %iowait    %irq   %soft   %idle    intr/s
07:44:18     all    3.01   57.31    0.36    0.13    0.01    0.00   39.19   1063.46
07:44:18       0    5.87   69.47    0.44    0.05    0.01    0.01   24.16    262.11
07:44:18       1    1.79   48.59    0.36    0.23    0.00    0.00   49.02    268.92
07:44:18       2    2.19   42.63    0.28    0.16    0.01    0.00   54.73    260.96
07:44:18       3    2.17   68.56    0.34    0.06    0.03    0.00   28.83    271.47

Report CPU utilization using the sar command

You can display today’s CPU activity, with sar command:
# sar
Output:

Linux 2.6.9-42.0.3.ELsmp (dellbox.xyz.co.in)         01/13/2007

12:00:02 AM       CPU     %user     %nice   %system   %iowait     %idle
12:10:01 AM       all      1.05      0.00      0.28      0.04     98.64
12:20:01 AM       all      0.74      0.00      0.34      0.38     98.54
12:30:02 AM       all      1.09      0.00      0.28      0.10     98.53
12:40:01 AM       all      0.76      0.00      0.21      0.03     99.00
12:50:01 AM       all      1.25      0.00      0.32      0.03     98.40
01:00:01 AM       all      0.80      0.00      0.24      0.03     98.92
...
.....
..
04:40:01 AM       all      8.39      0.00     33.17      0.06     58.38
04:50:01 AM       all      8.68      0.00     37.51      0.04     53.78
05:00:01 AM       all      7.10      0.00     30.48      0.04     62.39
05:10:01 AM       all      8.78      0.00     37.74      0.03     53.44
05:20:02 AM       all      8.30      0.00     35.45      0.06     56.18
Average:          all      3.09      0.00      9.14      0.09     87.68

Comparison of CPU utilization

The sar command writes to standard output the contents of selected cumulative activity counters in the operating system. The accounting system, based on the values in the count and interval parameters. For example display comparison of CPU utilization; 2 seconds apart; 5 times, use:
# sar -u 2 5
Output (for each 2 seconds. 5 lines are displayed):

Linux 2.6.9-42.0.3.ELsmp (www1lab2.xyz.ac.in)         01/13/2007

05:33:24 AM       CPU     %user     %nice   %system   %iowait     %idle
05:33:26 AM       all      9.50      0.00     49.00      0.00     41.50
05:33:28 AM       all     16.79      0.00     74.69      0.00      8.52
05:33:30 AM       all     17.21      0.00     80.30      0.00      2.49
05:33:32 AM       all     16.75      0.00     81.00      0.00      2.25
05:33:34 AM       all     14.29      0.00     72.43      0.00     13.28
Average:          all     14.91      0.00     71.49      0.00     13.61

Where,

  • -u 12 5 : Report CPU utilization. The following values are displayed:
    • %user: Percentage of CPU utilization that occurred while executing at the user level (application).
    • %nice: Percentage of CPU utilization that occurred while executing at the user level with nice priority.
    • %system: Percentage of CPU utilization that occurred while executing at the system level (kernel).
    • %iowait: Percentage of time that the CPU or CPUs were idle during which the system had an outstanding disk I/O request.
    • %idle: Percentage of time that the CPU or CPUs were idle and the system did not have an outstanding disk I/O request.

To get multiple samples and multiple reports set an output file for the sar command. Run the sar command as a background process using.
# sar -o output.file 12 8 >/dev/null 2>&1 &
Better use nohup command so that you can logout and check back report later on:
# nohup sar -o output.file 12 8 >/dev/null 2>&1 &

All data is captured in binary form and saved to a file (data.file). The data can then be selectively displayed ith the sar command using the -f option.
# sar -f data.file

Task: Find out who is monopolizing or eating the CPUs

Finally, you need to determine which process is monopolizing or eating the CPUs. Following command will displays the top 10 CPU users on the Linux system.
# ps -eo pcpu,pid,user,args | sort -k 1 -r | head -10
OR
# ps -eo pcpu,pid,user,args | sort -r -k1 | less
Output:

%CPU   PID USER     COMMAND
  96  2148 vivek    /usr/lib/vmware/bin/vmware-vmx -C /var/lib/vmware/Virtual Machines/Ubuntu 64-bit/Ubuntu 64-bit.vmx -@ ""
 0.7  3358 mysql    /usr/libexec/mysqld --defaults-file=/etc/my.cnf --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --socket=/var/lib/mysql/mysql.sock
 0.4 29129 lighttpd /usr/bin/php
 0.4 29128 lighttpd /usr/bin/php
 0.4 29127 lighttpd /usr/bin/php
 0.4 29126 lighttpd /usr/bin/php
 0.2  2177 vivek    [vmware-rtc]
 0.0     9 root     [kacpid]
 0.0     8 root     [khelper]

Now you know vmware-vmx process is eating up lots of CPU power. ps command displays every process (-e) with a user-defined format (-o pcpu). First field is pcpu (cpu utilization). It is sorted in reverse order to display top 10 CPU eating process.

iostat command

You can also use iostat command which report Central Processing Unit (CPU) statistics and input/output statistics for devices and partitions. It can be use to find out your system’s average CPU utilization since the last reboot.
# iostatOutput:

Linux 2.6.15.4 (debian)         Thursday 06 April 2006

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
     16.36    0.00    2.99    1.06    0.00   79.59

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
hda               0.00         0.00         0.00         16          0
hdb               6.43        85.57       166.74     875340    1705664
hdc               0.03         0.16         0.00       1644          0
sda               0.00         0.00         0.00         24          0 

You may want to use following command, which gives you three outputs every 5 seconds (as previous command gives information since the last reboot):$ iostat -xtc 5 3

GUI tools for your laptops/desktops

Above tools/commands are quite useful on remote server. For local system with X GUI installed you can try out gnome-system-monitor. It allows you to view and control the processes running on your system. You can access detailed memory maps, send signals, and terminate the processes.
$ gnome-system-monitor
Sample outputs:
gnome-system-monitor - view and control the processes
In addition, the gnome-system-monitor provides an overall view of the resource usage on your system, including memory and CPU allocation.

gnome-system-monitor - view and control the processes

Further readings

153 comment

  1. So you have 4 processors shown on your dual opteron machine which means that 2 of them are virtual. How do youfilter these out when you want to see only raw info on real processor utilization?

    Thanks

    Nikola

  2. I need to explain better i think:
    so i have 4 processors and what if only 2 are working and i still see 4 because of additional virtual processors? How do i check that? Plz answer its really important to me.

    Thanks in advance,

    Nikola

  3. If cpu entry not present in /proc/interrupts file your cpu is offline or not working for some causes:
    less /proc/interrupts

    You can also goto directory /sys/devices/system/cpu (note following commands needs special configuration option via kernel. If it is not complied it will not work for you):
    cd /sys/devices/system/cpu

    Type ls command to see all cpus
    ls

    Output:
    cpu0 cpu1 cpu2 cpu3 cpu4 cpu5 cpu6 cpu7

    Inside this directory you will see entry for online or offline CPU

    Hope this helps

  4. hi all i have a prob i use top and i have mysql has a %cpu 99.5 and i use mpstat but i have not the same results why with us and sy and ni
    i have smp linux rehl3

  5. Pádraig,

    Good syntax, just to list top 10, sorting should be -pcpu and pipe to head -10

    ps -e -o pcpu,cpu,nice,state,cputime,args –sort -pcpu | sed ‘/^ 0.0 /d’ | head -10

    Appreciate your post.

    1. [[email protected] topology]# ps -e -o pcpu,cpu,nice,state,cputime,args -sort -pcpu | sed ‘/^ 0.0 /d’ | head -10
      ERROR: Process ID list syntax err
      is it rt..Syntax or do we need to change any parameter.

      1. Syntax, two dashes before the sort argument, and single quotes around the sed expression – doesn’t copy and paste very well

  6. Ok, I can measure the cpu utilization… but what is the criteria to determine when a cpu is overloaded? Please let me know!

  7. Good Article. However, you neglected to mention vmstat. One of the nice things about vmstat is that it provides an insight into how the queues are filling up on each processor. Also, vmstat comes with just about every default install of UNIX and Linux that I have ever seen, so there are no additional files which need to be installed. vmstat is also very scriptable, if you need to log the cpu usage at intervals of time. Hope that this helps. Thanks again for a good article.

  8. Just FYI – on fedora machines, you’ll need to:

    $ yum install sysstat

    To install the sysstat package which contains (most of) the binaries listed in this article.

  9. yes its very good explaination..
    but how can one get this information like cpu usage into a program.. say C code?..
    say i wanted cpu usage into my float variable..
    please help.. thnks in advance…

  10. This is really a wonderful article…

    Learned a lot from this… And thank you all the readers for posting valuable comments with different commands.. this is really helpful for people like me.

    Diptanjan

  11. This is a nice and informative article.
    Can anyone suggest any command to monitor a process is using which cpu in SMP environment. I mean suppose I am running sshd. How do I know it is using cpu0 or cpu1 ?

  12. very good information…it would be good if we can have at least some hints on whether the stats shown is still ok or need to check on other items

  13. Really good article.

    I have a following question – what is the best way to find out which process(es) was(were) using cpu(s) for certain period of time based on cpu utilization threshold?

    Let’s imagine that we want to find all processes triggering CPU utilization beyond 70% for more then 10 seconds during last 24 hours and dump such an information in a log with the snapshots of time of occurrence, process ID and command.

    Anyone has an idea how to setup such a monitoring in an elegant way?

    Thanks,
    –invisible

  14. One of our server has four CPUs, but when I use mpstat or top, I can only see one CPU. In the directory /sys/devices/system/cpu, only show cpu0. But we do have 4 cpu, and OS admin told me all four cpu are runing. How do I know all four cpus are runing properly and I can use it.

    Thanks for any help.

  15. my server has four CPUs, but when I issue vmstat and top, I can only see one cpu. In the directory:/sys/devices/system/cpu, I can see only one cpu – cpu0. When I issue command:less /proc/interrupts, it only show one cpu. But there are do have four cpu on the server, should I do something to configure it, what should I do.

    Thanks for any help.

  16. >should I do something to configure it, what should I do

    Yup, ask your admin to install Linux SMP kernel and boot into the same using Grub. Once booted using SMP, run uname -a to verify that SMP kernel loaded. Also go through /proc/cpuinfo file.
    less /proc/cpuinfo

  17. The Linux Admin told me kernel-smp are package for free linux and some old version, we runing advanced linux version and do not need it, it confused me. Is it true?

  18. Thanks for all the help.

    It is clear now, we did not install smp linux kernel, so there is only one cpu working. Could any one tell me when install Redhat linux on multip cpu server, it will automatically configure to smp kernel or have to maually select the kernel?

  19. zhili,

    In most cases it is installed by default but sometime installer cannot detect it and it will install normal kernel. Run yum / up2date command to install SMP kernel from RHN.

  20. Hi, I have a question about iostat command. How can I see the %utilization of disk not %utilization of CPU? I have found out from some of the forums and man page for iostat that the disk utilization report ought to show one parameter called %util. And also somewhere I have found a flag -D to show it but I cannot find anywhere how to configure my command so that it shows this %util field.

    Any help is highly appreciated. Thank you.

  21. Daton,

    Sometime man page can be confusing..just run iostat -d -x to display disk utilization including %util. It will show TPS (number of transfers per second ) and amount of data read and/or wrtite to/from the device. Try following examples,
    iostat -d -x
    iostat -d -m -x
    iostat -x -d 2

    All you have to do is pass -x option.

  22. when we see that certain process takes 10% of CPU and we have 2 CPU server, does it mean – it is 10% from one CPU (==5% of server CPU power ) or 10% from server CPU power (==20% of one CPU)

  23. HI nice tute!

    Same question as Vitaly above.

    Ina 2 cpus enviroenmnt does the cpu usage percentage mean the percentage the process takes of 2 cpu or both. Please see the above exmple by Vitaly.

    Also in your image of the top output. The cpu usage vales for the processes add up to a value (26.8%) greater than the total cpu usage value 23.%)

  24. Great intro to CPU usage. Thanks.

    The top 10 monopolizing process would be better if sorted as a numberic key :


    ps -eo pcpu,pid,user,args | sort -k 1 -r -n | head -10

  25. I have a dual core system. When I type ‘mpstat -P ALL’

    10:52:18 PM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
    10:52:18 PM all 14.33 0.13 2.49 0.43 0.03 0.18 0.00 82.41 151.04
    10:52:18 PM 0 15.47 0.13 2.47 0.33 0.00 0.01 0.00 81.59 0.00
    10:52:18 PM 1 13.21 0.13 2.51 0.53 0.07 0.35 0.00 83.21 151.04
    10:52:18 PM 2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

    What is CPU 2?

  26. Hi All,

    Today is 5/28/2008 but I ran some performance tests during 5/7/2008.I need to capture the measurements for both Memory and CPU using ‘SAR’ commands for each of 1 minute interval. Is it possible? If yes, what commands should I need to use?

    I appreciate your input here.

    Thanks
    Satya

  27. Thanks for writing this, it’s a very good document about how to use proc. Excellent work. Things like this should be in the kernel docs but they just aren’t. Thanks for filling in the gaps.

  28. hmmm. how will i make use of the sourcecode used by top command in calling the program to display the processes, to be used in my bankers program in C, to test how bankers handle thing in real time…thanks

  29. Thanks for the mpstat command. I have a question on how to interpret the CPU usage output from all those commands (top, mpstat, etc)

    As from my understanding, when a core is doing stuffs, it is fully occupied, i.e. always 100%. So, is the CPU usage meaning the % use of this CPU since the last reboot? Or is it meaning the % use within a timeframe?

    Thanks

  30. Just to add information to my question above. For mpstat, there is a INTERVAL param. If it is set as 2 seconds, does the output mean the average % CPU usage just within that 2 seconds timeframe?

    Thanks.

  31. Is there a way to query the overall system usage say in a Perl script or something? I have a low power Linux (or XP it could be) box and want it to postpone some activity for 5 minutes if the CPU loading is say over 30%.

    In simple terms

    If CPU Usage > 30% then Wait 300 Seconds

    Can that be done?-

  32. Vivek,

    Great article!

    I have a question. I am confused by “%CPU” in top. One user process used 100% CPU, but the system still have 89% idle, not 0% idle. How to understand it?

    Regards,
    Louis

    $ top |head -8
    top – 12:50:45 up 33 days, 19:32, 4 users, load average: 1.06, 1.05, 1.00
    Tasks: 234 total, 2 running, 229 sleeping, 1 stopped, 2 zombie
    Cpu(s): 9.1%us, 1.7%sy, 0.0%ni, 89.0%id, 0.1%wa, 0.0%hi, 0.1%si, 0.0%st
    Mem: 3894760k total, 3708140k used, 186620k free, 239680k buffers
    Swap: 2031608k total, 17700k used, 2013908k free, 2374436k cached

    PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
    3595 otrs 25 0 26712 21m 3840 R 99 0.6 17931:35 PostMasterMailb

    $ mpstat -P ALL
    Linux 2.6.18-8.el5 (xxxxxxxxx) 12/31/2008

    12:54:42 PM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
    12:54:42 PM all 9.09 0.00 1.67 0.07 0.01 0.13 0.00 89.03 1018.11
    12:54:42 PM 0 1.93 0.00 0.31 0.11 0.00 0.03 0.00 97.62 255.65
    12:54:42 PM 1 9.78 0.00 1.68 0.04 0.01 0.13 0.00 88.37 254.53
    12:54:42 PM 2 2.48 0.00 0.40 0.11 0.00 0.03 0.00 96.97 253.35
    12:54:42 PM 3 22.16 0.00 4.31 0.02 0.01 0.35 0.00 73.16 254.58

  33. Hi Vivek,

    that is a goos article. hope you can help me in this small issue.
    i have got lighttpd running and i need to develop test cases and run tests – provide dynamic content with HTML. use options like cgi, fastcgi, ajax etc.

    i want to monitor the resource usage of lighttpd during each test.

    please do reply. thanks

  34. Hi all
    It’s a nice article
    I want to relate the CPU utilization with the energy consumed by the ststem, is there any way to do so.
    Or there is an alternate way to find the energy consumed by all the processes.

    Regards
    Youvedeep Singh

  35. Hey all! Nice article
    Can anyone tell me where is this information stored? For example in unix systems at /proc/uptime gives information of uptime of the system and the value is dynamic since it would be different every time you poll. Which file stores such information for CPU utilization? So I can “cat and get the information that how much CPU has been utilized?
    Any help is appreciated.
    Best Regards
    Vikrant

    1. You can try to use /proc/stat to calculate CPU utilization.

      If the information is displayed like this:
      cpu 607557 143290 385125 136873807 1279793 18764 14856 0 0

      To get the average system load over any given amount of time you read only the first four values into variable (for example, ill call them u1, n1, s1, and i1). Then when you’re ready you read the values again into new variables, u2, n2, s2, and i2. Now your total usage time is equal to (u2-u1) + (n2 – n1) + (s2 – s1). And your total time overall is the usage time + (i2-i1). Take (100*usage)/total and you have your percent CPU Usage.

  36. Can someone please explain me how can it be when /proc/stats shows 414% total CPU usage (including the idle time) on a quad-core single processor box?

    Example:

    $ cat /proc/stat|grep “^cpu “;sleep 100;cat /proc/stat|grep “^cpu ”
    cpu 102843 0 66548 4308888 773304 9547 30153 0 0
    cpu 103568 0 67109 4345597 776311 9655 30471 0 0

    if we subtract the sets of numbers from each other, divide by 100 and sum them, we’ll find that they add up to 414.28. How can it be? I use RHEL 5 with 2.6.28.2 kernel with dynamic ticks and multi-core scheduler support.
    Thanks.

    1. The average CPU usage is:

      (103568-102843 + 0-0 + 67109-66548)/(4345597-4308888)
      =0.0350322809

      only 3.5%, but the estimation is not accurate

  37. Found it out. The bug (feature?) was in the CONFIG_NO_HZ option. I guess the kernel calculates the idle time inaccurately when the dynamic ticks are turned on. However I also have a few other 32-bit boxes (that one was 64-bit) where this issue does not occur and one 32-bit where it does.
    Weird.

  38. hi

    In one of my college project i require the energy consumed by the System.
    Is there any way to find the Power/Energy consumed by the CPU/Disk/IOs.
    If anyone is aware of any of such method, please reply fast, i require it urgently.

    Regards
    Youvedeep Singh

  39. How I get the word size (32bit or 64bit) of a particular server running on Linux
    Linux express.sagt.com.lk 2.6.9-34.ELsmp #1 SMP Fri Feb 24 16:54:53 EST 2006 i686 i686 i386 GNU/Linux

    Thanks
    Nic

  40. Using “k -pcpu” will accomplish the reverse sorting as well (including taking care of proper numeric sorting on multi-cpu machines):
    ps k -pcpu -eo pcpu,pid,user,args | head -10

  41. hi hi well…. i have 1question.. ahmm….letme se…. ihave a Red hat 5 linux and glassfish aplication server an’ when i try to monitoring the server i see something like …. well ….

    top – 17:41:08 up 1 day, 6:37, 4 users, load average: 1.12, 1.12, 1.11
    Tasks: 159 total, 1 running, 153 sleeping, 5 stopped, 0 zombie
    Cpu(s): 26.5%us, 0.2%sy, 0.0%ni, 73.1%id, 0.0%wa, 0.0%hi, 0.2%si, 0.0%st
    Mem: 3365124k total, 3124404k used, 240720k free, 4400k buffers
    Swap: 8385888k total, 246344k used, 8139544k free, 71892k cached

    PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
    12001 root 20 0 1633m 593m 11m S 102 18.0 218:19.97 java
    28432 root 22 0 1641m 330m 13m S 2 10.1 2:06.12 java
    28611 root 25 0 1642m 228m 12m S 2 6.9 2:25.95 java
    28528 root 19 0 2253m 920m 13m S 1 28.0 10:43.70 java
    1 root 15 0 2036 556 524 S 0 0.0 0:00.84

    as u’ can see mi cpu it’s burning or something like that …. someone knows why a glassfish domain only increese i mind…just eat ram memory…. someone knows.. why….. jus whyy !!! why!!!!! T_T

  42. ahmm….. really?….. well…. ahm…. the server in the graphic mode runs to slowly and welllit’s a little server i think cuz just have 3.5gb in Ram , but as u’ can se in my example the proces quickly the ram downs……i mind… i don’t know i think……

  43. Good info here. I like the “top|head -5” in particular – everything in one nutshell. Can someone please explain the CPU load percentage as it relates to multi-core or multi-cpu machines. If I see a load of 1.57 (157%) on a Xeon (quad processor), is that 1.57 out of a possible 4.00, or are all 4 CPUs running flat out and swapping the 57% overhead? Also, is there an easy way to get a CPU count to adjust the load statistic if needed? From the above the best I got was “cat /proc/cpuinfo|grep processor|wc -l” – but I don’t suspect it would ‘port’ from RedHat to a Sun, HP or AIX box.

  44. Show me a command that would search for a given line of text in all files in the current directory tree older than 2 months without using pipes

  45. While looking in an application log file, you see a line to the effect of “Cannot bind to address: 1.2.3.4 address already in use”; what does this indicate?

  46. ps can also sort the output

    ps -eo pcpu,vsize,rss,pid,user,args –sort -pcpu

    i am using this now in a monitoring script so as soon as something goes wrong i can get a snapshot of the cpu hog, and check up on process memory usage as well.

  47. I have query here. Appreciate if any one could help me on this.

    My server is ProLiant DL380 G3 model which is a 32 bit CPU server. It has two cpus. Now the problem is server alsways showing high IOWAIT.

    CPU states: cpu user nice system irq softirq iowait idle
    total 3.2% 0.0% 0.5% 0.0% 0.0% 95.7% 0.3%
    cpu00 1.1% 0.0% 0.7% 0.0% 0.0% 98.0% 0.0%
    cpu01 5.3% 0.0% 0.3% 0.0% 0.0% 93.4% 0.7%

    There is no process which is taking high CPU. All the process are looks normal. And the total cpu utilization also showing very less. Its started two month back and since then the server performance is too slow.

    Can any one help me on this please?

  48. While using top press “1” can show load of each cpu(core) as shown below

    op – 20:10:48 up 2 days, 1:30, 1 user, load average: 1.01, 1.01, 0.97
    Tasks: 119 total, 2 running, 117 sleeping, 0 stopped, 0 zombie
    Cpu0 : 0.0% us, 0.0% sy, 0.0% ni, 99.7% id, 0.3% wa, 0.0% hi, 0.0% si
    Cpu1 : 99.7% us, 0.3% sy, 0.0% ni, 0.0% id, 0.0% wa, 0.0% hi, 0.0% si
    Cpu2 : 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0% si
    Cpu3 : 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0% si
    Cpu4 : 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0% si
    Cpu5 : 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0% si
    Cpu6 : 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0% si
    Cpu7 : 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0% si
    Mem: 32905096k total, 20045912k used, 12859184k free, 264608k buffers
    Swap: 34812844k total, 0k used, 34812844k free, 5588896k cached

  49. Guys Please let me know When I give
    # mpstat -P ALL
    I can see 16 CPU’s [0 to 15].
    How Can I check How many physical CPU’s are installed on server

  50. Hi Guys!

    Just wanted to offer you to email me, if you are having trouble to configure/monitor your system performance on linux.
    I am working as a gameserver admin, and I have been using a lot of different linux-versions, including SuSE, RedHat, Debian, Gentoo and some other Unix-like systems.
    As you can imagine, Realtime-Online-Gameservers are very dependent on a healthy and FAST system, where CPU,RAM & IO – Usage is extremely important, because gameservers would lag otherwise..
    During the past few years, I found out a lot of tricks, read hundreds of howtos, and compiled many many different kernels with custom features for a lot of different hardware, include 32bit-single-cpus, 64bit-multicore-cpus, ppc-architecture-systems and so on and so on..
    So if you have got real trouble, which you can`t easily google yourself, let me know, I`ll be happy to help out..
    Best regards, Oliver

    1. Hi,
      i have a query regarding CPU utilization and CPU percentage.
      What is mean by CPU percentage?
      Basing on what parameter we can calculate CPU percentage.
      On my AIX server having 4 cpu’s shows below values for each 15 min interval.

      system cpu time is 4%, User cpu time is 90% idle cpu time is 6%

      and CPU percentage of all the processes not exceeded 20%.
      How can i understood my CPU performance from the above values.

  51. When on a multicore system, does top with no arguments show an average of each core? I know that with the “1” argument I can see each CPU listed. But with no arguments what am I seeing? It appears to be an average of stats on each CPU?

    Thanks! Craig

  52. Hi Oliver Kleinecke,
    I have e-commerce site.
    which is using two load balancer and 8 web servers.
    how can I monitor server performance of each machine or CPU.
    Thanks,
    Prashant

  53. Thanks much for all this info,
    is it possible to get daily stats written into a log file?
    or do I need to get some script to do that?
    any recommendations?
    thank you
    MM

    1. Hi,

      Is there a way to find out cpu usage per thread in Linux? Some other threads say that we cannot find cpu usage at thread level in Linux? Is that true? Any material to give more insight on this topic.

      I know /proc//lpsinfo structure gives cpu usage per thread in Solaris.

      Appreciate your response,
      Thanks, Venkat

  54. Here there is a piece of a script that may be useful to get the CPU usage only:

    top bd00.50n2 | grep Cpu | tail -n1 | sed ‘s/.*Cpu(s)://g; s/us,.*//g’

    This gives the User CPU usage, but could be easily modified for other values.

    1. How to run the following script…please help…

      top bd00.50n2 | grep Cpu | tail -n1 | sed 's/.*Cpu(s)://g; s/us,.*//g'

      I am using ubuntu.

  55. Hi ,
    I am running mpstat -P ALL command and calculating the CPU utilization values. but i want to calculate the cpu utilization values of another CPU from the one that I am working from and report the values of the same to other module that is working on in my processor ..Is it possible? Please let me know ..This is a urgent requirement..

    1. There’s no such thing as “urgent requirement” when you don’t compensate people in some way. And when you do, spelling things clearly in the first place helps a lot: having read your message twice I can’t see any impementable solution since there are too many possible interpretations.

  56. Great article!!! thanks
    although i have a debian squeeze box without X GUI installed although i’d like to have graphical reports on performance statistics like CPU%, memory usage etc.. is there any tool or command i can use to get this done? using systat i noticed im getting volumes of text data and its quite time consuming to search for specific data. im quite new to this could anyone help me out

    Thanks in advance

  57. to be more specific i was using *sar and found all the data i wanted and more but realized that i needed more Graphical data. could anyone suggest which direction i need to move to get this done?

  58. Is there a way for me to see sar results for yesterday from the default log?
    ( By default it only shows from midnight of today . Note: I am not explicitly running or saving it )

    Thanks .

  59. Could you kindly descripe more detail about the below command. For example, if use command number 2 “nohup”, what directory should i used? Local directory or server directory? Thank in advance for your kind.

    To get multiple samples and multiple reports set an output file for the sar command. Run the sar command as a background process using.
    # sar -o output.file 12 8 >/dev/null 2>&1 &
    Better use nohup command so that you can logout and check back report later on:
    # nohup sar -o output.file 12 8 >/dev/null 2>&1 &
    All data is captured in binary form and saved to a file (data.file). The data can then be selectively displayed ith the sar command using the -f option.
    # sar -f data.file

  60. Nice article and equally good comments suggesting the new commands.
    I can see some sar files inside /var/log/sa/. I can see a block containing the CPU %user %nice %sys %iowait %irq %soft %steal %idle values. Is there some way to see memory utilization in the same file, memory utilization per process or as whole.
    Please help

  61. Hi,

    this is my output from iostat command:

    Linux 3.0.0-14-generic (k1-desk) 	01/03/2012 	_x86_64_	(4 CPU)
    
    avg-cpu:  %user   %nice %system %iowait  %steal   %idle
               3.67    0.02    1.51    0.62    0.00   94.18
    
    Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
    sda              31.60     14245.90        85.57  979239785    5881884
    sdb              30.24         0.01     14285.26        976  981945348
    sdc               1.47        20.83        18.15    1431840    1247465
    dm-0              5.08        37.71        85.57    2592093    5881884
    dm-1              4.96        37.70        85.57    2591693    5881884
    

    this is very interesting because sda and sdb are same disks in mirror raid, lower named dm-1.. does this mean that disks in my raid work that way,,, one reads data, other write it ? … strange.. but makes my head think about that :)

    1. @mkdizajn,

      I would guess from the numbers that sda and sdb are 1TB drives. Sine it appears to have read 1TB from sda and written it to sd, mygues is t thiis the mirrorin process. After yreboo, the nunbs shod even out.The total reads on sda and sdb should add to dm-1 and the writes should be the same.

  62. SO LETS SAY I FIND OUT WHAT IS USING ALL MY CPU AND KEEPING IT RUNNING AT 100%. I FIND OUT THAT IT IS NOT A PROCESS THAT I START UP KNOWINGLY. HOW DO I STOP IT FROM LOADING ON NEXT REBOOT? MY COMPUTER RUNS SO SLOW DUE TO THE CPU RUNNING AT 100% CONTINUALLY. I WANT TO FIX THIS. AT FIRST I THOUGH IT WAS DUE TO INSUFFICIENT MEMORY SO I REMOVED WINDOWS AND INSTALLED LINUX. I ALSO DOUBLED MY MEMORY. NOW I AM AT 512MB BUT IT STILL RUNS THE SAME AS IT DID WITH 256MB.

    1. Hi Michael,

      Typing in CAPS takes more memory than not. That’s why when you hit the shift button the CPU spikes and the memory shoots up just for a second. I believe by the context of your message that you are unknowingly forcing you’re RAM and CPU to shoot up by keeping the CAPS lock button on, or rather holding SHIFT while you type.

      Please try reinstalling Windows and let us know your results.

  63. Great article. Is there a way to get all the counters in the row instead of the header? f.ex.:
    Date, Time, HOST, OBJECTNAME, COUNTER, INSTANCE
    2012-09-18,12:11,TEST,CPU,%IDLE,_ALL
    2012-09-18,12:11,TEST,DEVICE,TPS,_SDA
    2012-09-18,12:12,CPU,%system,_AVG

    This should be used for collecting data for many hosts into an datawarehouse. The idea is to create an performance hierachy for different performancecounters like CPU,MEM,DISK aso…

  64. Please correct your sort command adding -n (numeric)
    Exemple : sort -r -k1 should be sort -nr -k1
    ps -eo pcpu,pid,user,args | sort -nr -k1 | less

    Otherwise 7% is before 30%

  65. Hi,

    Just know that ps show CPU from since the process started, not like for example top, which show the current CPU usage status.
    Therefore, don’t trust “ps” in case you need your current CPU status per process ID.

  66. Hello

    I am wondering why my cpu folder is empty. I got this error from my KVM, cannot open /sys/devices/system/cpu/cpu0/topology/thread_siblings: no such file or directory. I am using CentOS (VPS from Hostgator).

  67. Hi,

    If wanted to find out which process took high CPU 3 days back any command available? Also MySql DB is showing 738.5% of CPU, is that a real number, if it is real what is the reason for that?

    top – 19:04:06 up 176 days, 21:04, 2 users, load average: 1.13, 1.15, 1.16
    Tasks: 532 total, 1 running, 531 sleeping, 0 stopped, 0 zombie
    Cpu(s): 28.2%us, 4.6%sy, 0.0%ni, 67.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
    Mem: 148693532k total, 110977488k used, 37716044k free, 2422156k buffers
    Swap: 8388600k total, 482248k used, 7906352k free, 96894392k cached

    PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
    5809 mysql 20 0 12.2g 133m 4548 S 738.5 0.1 958656:57 mysqld
    16443 vzbrprod 20 0 5669m 382m 17m S 28.7 0.3 6642:07 java
    23198 vzbrprod 20 0 32.7g 286m 12m S 26.0 0.2 601:33.03 java
    13909 vzbrprod 20 0 35.9g 197m 12m S 10.5 0.1 1137:23 java
    11108 v977139 20 0 15432 1596 936 R 1.3 0.0 0:00.09 top
    109 root 20 0 0 0 0 S 0.3 0.0 9:22.85 events/10
    2609 root 20 0 9136 560 400 S 0.3 0.0 154:03.55 irqbalance
    3744 root 20 0 39008 2072 1392 S 0.3 0.0 14:01.15 selogrd

  68. Hi

    Very useful article

    Is it possible to get stats information like “prstat -t2 provides, I mean, to report total usage summary for each user. Example:
    NPROC USERNAME SWAP RSS MEMORY TIME CPU
    78 casuser 16G 14G 22% 4913:40:2 8.1%
    56 root 3635M 3743M 5.7% 1995:40:4 2.1%
    1 nobody 1200K 4040K 0.0% 0:00:01 0.0%
    1 noaccess 132M 112M 0.2% 13:25:20 0.0%
    7 xoifo 5296K 15M 0.0% 0:00:00 0.0%
    1 smmsp 2576K 10M 0.0% 0:05:33 0.0%
    6 daemon 9792K 12M 0.0% 1:12:41 0.0%

Leave a Comment