Linux pidstat Command: Get Statistics For Individual Tasks / Process

Posted on in Categories Command Line Hacks last updated August 21, 2012

Process identifier (PID) is a number used by Linux / Unix kernels (and Windows operating systems) to identify a process. Usually, new processes are created using the fork() system call. Each PID (or so called tasks) can be monitored under Linux. In this quick tutorial, I will explain how to use the pidstat command for monitoring individual tasks currently being managed by the Linux kernel.

The pidstat command can also be used for monitoring the child processes of selected tasks and it can answer the following questions for you:

  1. See stats for a particular process.
  2. See disk stats for a particular process (e.g., disk I/O info for Apache/Nginx/Lighttpd and all of it’s child processes).
  3. See statistics for threads associated with selected tasks.
  4. See reports of CPU statistics for every active task.
  5. Find out: how many context switches a process makes?
  6. Find out: page faults and memory utilization for a particular process?
  7. Confirm memory leaks for a particular process.
  8. And much more.

Syntax

The syntax is:

pidstat
pidstat options interval count

Type the following command:
# pidstat
Sample outputs:

05:14:33 PM       PID    %usr %system  %guest    %CPU   CPU  Command
05:14:33 PM         1    0.00    0.00    0.00    0.00     2  init
05:14:33 PM         3    0.00    0.00    0.00    0.00     0  migration/0
05:14:33 PM         4    0.00   12.98    0.00   12.98     0  ksoftirqd/0
05:14:33 PM         5    0.00    0.00    0.00    0.00     0  migration/0
05:14:33 PM         6    0.00    0.00    0.00    0.00     0  watchdog/0
05:14:33 PM         7    0.00    0.00    0.00    0.00     1  migration/1
05:14:33 PM         8    0.00    0.00    0.00    0.00     1  migration/1
05:14:33 PM         9    0.00    5.70    0.00    5.70     1  ksoftirqd/1
05:14:33 PM        10    0.00    0.00    0.00    0.00     1  watchdog/1
...
...
..05:14:33 PM      7915    0.00    0.00    0.00    0.00     1  nfsd4
05:14:33 PM      7917    0.00    0.01    0.00    0.01     6  nfsd
05:14:33 PM      7918    0.00    0.01    0.00    0.01     6  nfsd
05:14:33 PM      7919    0.00    0.01    0.00    0.01     6  nfsd
05:14:33 PM      7920    0.00    0.02    0.00    0.02     1  nfsd
05:14:33 PM      7921    0.00    0.01    0.00    0.01     4  nfsd
05:14:33 PM      7922    0.00    0.01    0.00    0.01     6  nfsd
05:14:33 PM      7923    0.00    0.02    0.00    0.02     4  nfsd
05:14:33 PM      7924    0.00    0.01    0.00    0.01     5  nfsd
05:14:33 PM      9254    0.00    0.00    0.00    0.00     6  php-cgi
05:14:33 PM      9255    0.00    0.00    0.00    0.00     4  php-cgi
05:14:33 PM      9266    0.00    0.00    0.00    0.00     4  php-cgi
05:14:33 PM     12767    0.00    0.00    0.00    0.00     6  php-cgi
05:14:33 PM     12776    0.00    0.00    0.00    0.00     6  php-cgi
05:14:33 PM     12784    0.00    0.00    0.00    0.00     6  php-cgi
05:14:33 PM     14800    0.00    0.00    0.00    0.00     6  php-cgi
05:14:33 PM     14832    0.00    0.00    0.00    0.00     6  php-cgi
05:14:33 PM     14836    0.00    0.00    0.00    0.00     4  php-cgi
05:14:33 PM     14837    0.00    0.00    0.00    0.00     6  php-cgi
05:14:33 PM     15016    0.00    0.00    0.00    0.00     6  sshd
05:14:33 PM     15019    0.00    0.00    0.00    0.00     0  bash
05:14:33 PM     15089    0.00    0.00    0.00    0.00     4  php-cgi
05:14:33 PM     15090    0.00    0.00    0.00    0.00     4  php-cgi
05:14:33 PM     15091    0.00    0.00    0.00    0.00     6  php-cgi

See stats I/O stats from any process

To see I/O statistics (kernels 2.6.20 and later only) for nfsd4 process run by PID # 13212, enter:
# pidstat -d -p 13212
# pidstat -d -p 13212 2 10
# pidstat -d -p 7114 1 50
Sample outputs (from mysqld server w/ PID # 7114):

Linux 2.6.32-279.el6.x86_64 (server1.cyberciti.biz) 	08/21/2012 	_x86_64_	(8 CPU)
 
05:24:35 PM       PID   kB_rd/s   kB_wr/s kB_ccwr/s  Command
05:24:36 PM      7114      0.00     40.00      0.00  mysqld
05:24:37 PM      7114      0.00     64.00     64.00  mysqld
05:24:38 PM      7114      0.00     44.00      8.00  mysqld
05:24:39 PM      7114      0.00     24.00      0.00  mysqld
05:24:40 PM      7114      0.00    128.00    128.00  mysqld
05:24:41 PM      7114      0.00     88.00     64.00  mysqld
05:24:42 PM      7114      0.00     92.00      0.00  mysqld
05:24:43 PM      7114      0.00    152.00     64.00  mysqld
05:24:44 PM      7114      0.00     48.00      0.00  mysqld
05:24:45 PM      7114      0.00     44.00      0.00  mysqld
05:24:46 PM      7114      8.00   1140.00     32.00  mysqld
05:24:47 PM      7114      8.00  20124.00   3192.00  mysqld
05:24:48 PM      7114      0.00   1992.00   2956.00  mysqld

You will see the following values:

  • PID – The identification number of the task being monitored.
  • kB_rd/s – Number of kilobytes the task has caused to be read from disk per second.
  • kB_wr/s – Number of kilobytes the task has caused, or shall cause to be written to disk per second.
  • kB_ccwr/s – Number of kilobytes whose writing to disk has been cancelled by the task. This may occur when the task truncates some irty pagecache. In this case, some IO which another task has been accounted for will not be happening.
  • Command – The command name of the task.

Find out top 5 process page faults statistics

The following command will show you five reports of page faults statistics at two second intervals for the child processes of all tasks in the system:
# pidstat -T CHILD -r 2 5
Sample outputs:

Linux 2.6.32-279.el6.x86_64 (server1.cyberciti.biz) 	08/21/2012 	_x86_64_	(8 CPU)
 
05:27:12 PM       PID minflt-nr majflt-nr  Command
05:27:14 PM      1853        10         0  irqbalance
05:27:14 PM      7114         3         0  mysqld
05:27:14 PM      7910         2         0  rpc.idmapd
05:27:14 PM     15089         1         0  php-cgi
05:27:14 PM     15090         1         0  php-cgi
05:27:14 PM     15091         2         0  php-cgi
05:27:14 PM     15361         1         0  php-cgi
05:27:14 PM     15362         1         0  php-cgi
05:27:14 PM     15395       704         0  pidstat
 
05:27:14 PM       PID minflt-nr majflt-nr  Command
05:27:16 PM      7114       169         0  mysqld
05:27:16 PM     15359         1         0  php-cgi
05:27:16 PM     15360        21         0  php-cgi
05:27:16 PM     15361         2         0  php-cgi
05:27:16 PM     15395       705         0  pidstat
 
05:27:16 PM       PID minflt-nr majflt-nr  Command
05:27:18 PM      7114        14         0  mysqld
05:27:18 PM     15362         2         0  php-cgi
05:27:18 PM     15395       702         0  pidstat
 
05:27:18 PM       PID minflt-nr majflt-nr  Command
05:27:20 PM      7114       162         0  mysqld
05:27:20 PM     15359         2         0  php-cgi
05:27:20 PM     15360         2         0  php-cgi
05:27:20 PM     15395       702         0  pidstat
 
05:27:20 PM       PID minflt-nr majflt-nr  Command
05:27:22 PM      7114        16         0  mysqld
05:27:22 PM     15360         2         0  php-cgi
05:27:22 PM     15361         2         0  php-cgi
05:27:22 PM     15362         1         0  php-cgi
05:27:22 PM     15395       702         0  pidstat
 
Average:          PID minflt-nr majflt-nr  Command
Average:         1853         2         0  irqbalance
Average:         7114        73         0  mysqld
Average:         7910         0         0  rpc.idmapd
Average:        15089         0         0  php-cgi
Average:        15090         0         0  php-cgi
Average:        15091         0         0  php-cgi
Average:        15359         1         0  php-cgi
Average:        15360         5         0  php-cgi
Average:        15361         1         0  php-cgi
Average:        15362         1         0  php-cgi
Average:        15395       703         0  pidstat

Where,

  • PID – The identification number of the task which is being monitored together with its children.
  • minflt-nr – Total number of minor faults made by the task and all its children, and collected during the interval of time.
  • majflt-nr – Total number of major faults made by the task and all its children, and collected during the interval of time.

See CPU utilization

Type the following command:
# pidstat -T CHILD -u 2 2
Sample outputs:

Linux 2.6.32-279.el6.x86_64 (server1.cyberciti.biz) 	08/21/2012 	_x86_64_	(8 CPU)
 
05:32:50 PM       PID    usr-ms system-ms  guest-ms  Command
05:32:52 PM         4         0       410         0  ksoftirqd/0
05:32:52 PM      7114        40        10         0  mysqld
05:32:52 PM     15424        10        30         0  pidstat
05:32:52 PM     48440         0        10         0  memcached
 
05:32:52 PM       PID    usr-ms system-ms  guest-ms  Command
05:32:54 PM         4         0       220         0  ksoftirqd/0
05:32:54 PM      7114        10        20         0  mysqld
05:32:54 PM     15424        10        20         0  pidstat
05:32:54 PM     48440        10         0         0  memcached
 
Average:          PID    usr-ms system-ms  guest-ms  Command
Average:            4         0       315         0  ksoftirqd/0
Average:         7114        25        15         0  mysqld
Average:        15424        10        25         0  pidstat
Average:        48440         5         5         0  memcached

When reporting global statistics for tasks and all their children, the following values are displayed:

  1. PID – The identification number of the task which is being monitored together with its children.
  2. usr-ms – Total number of milliseconds spent by the task and all its children while executing at the user level (application), with or without nice priority, and collected during the interval of time. Note that this field does NOT include time spent running a virtual processor.
  3. system-ms – Total number of milliseconds spent by the task and all its children while executing at the system level (kernel), and collected during the interval of time.
  4. guest-ms – Total number of milliseconds spent by the task and all its children in virtual machine (running a virtual processor).
  5. Command – The command name of the task which is being monitored together with its children.

This command has many more options, please go through man page for more info:
$ man pidstat

Recommended readings
  • iostat Command – See Central Processing Unit (CPU) statistics and input/output statistics for devices, partitions and network filesystems (NFS)
  • mpstat Command – See CPU (processors) related statistics.
  • sar command and kSar tool – Identifying Linux server bottlenecks with sar and kSar.
  • man pages – pidstat(1),top(1), ps(1), and vmstat(8)

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin and a trainer for the Linux operating system/Unix shell scripting. He has worked with global clients and in various industries, including IT, education, defense and space research, and the nonprofit sector. Follow him on Twitter, Facebook, Google+.

Share this on:

Tagged as: Tags ,

4 comment

  3. It appears that you’re using RHEL or CentOS 5. You can find out which package supplies it by running this:

    yum whatprovides “*/pidstat”

    This should lead you to the sysstat package.

    Reply Report comment

  4. hi
    on my linux distro : Linux localdomain 2.6.18-128.1.10.el5 #1 SMP Thu May 7 10:39:21 EDT 2009 i686 i686 i386 GNU/Linux
    pidstat cannot be found; how/where do you install it etc .. ?

    Reply Report comment

Leave a Comment