Linux: Find Out What’s Using Up All Virtual Memory

How do I list or display all processes, executables, and shared libraries that are using up all virtual memory under Debian or Ubuntu Linux server for troubleshooting purpose?

Virtual memory is nothing but a memory management technique used by Linux and many other modern operating systems.

Tutorial details
Difficulty	Easy
Root privileges	Yes
Requirements	memstat tool

It combines active RAM and inactive memory on disk drive (hard disk or ssd and so on) to form a large range of contiguous addresses. You can use the memstat command to see all processes, executables, and shared libraries that are using up virtual memory. This command must be run as root user as you need to see the data of all running processes. The output is intended to help you identify programs that are using an excessive amount of memory, and to reduce overall memory waste. The syntax is as follows:

memstat
memstat -w
memstat -p PID-HERE

The -w option causes a wide printout: lines are not truncated at 80 columns. The -p option causes memstat to only print data gathered from looking at the process with the given PID (such as an Apache process.

Examples

Type the following command:
# memstat
OR
$ sudo memstat
Sample outputs (from my home nfs server):

    272k: PID     1 (/sbin/init)
    536k: PID   467 (/sbin/udevd)
    532k: PID   586 (/sbin/udevd)
    276k: PID  1191 (/sbin/portmap)
    296k: PID  1204 (/sbin/rpc.statd)
    308k: PID  1222 (/usr/sbin/rpc.idmapd)
    268k: PID  1385 (/usr/sbin/vnstatd)
  91028k: PID  1424 (/usr/sbin/rsyslogd)
    416k: PID  1473 (/sbin/mdadm)
   1068k: PID  1493 (/usr/bin/atop)
    464k: PID  1517 (/usr/sbin/atd)
    404k: PID  1520 (/usr/sbin/rpc.mountd)
    476k: PID  1530 (/usr/sbin/cron)
    300k: PID  1539 (/usr/sbin/acpid)
    560k: PID  1545 (/usr/sbin/sshd)
    300k: PID  1550 (/usr/bin/dbus-daemon)
    276k: PID  1568 (/sbin/getty)
  78404k: PID  1606 (/usr/sbin/console-kit-daemon)
    732k: PID  1755 (/usr/sbin/sshd)
   2160k: PID  1760 (/bin/bash)
    292k: PID  1807 (/usr/bin/ssh-agent)
    532k: PID  1942 (/sbin/udevd)
    776k: PID  1944 (/usr/sbin/pppd)
    276k: PID  1945 (/bin/dash)
    280k: PID  1948 (/usr/sbin/pptp)
    280k: PID  1956 (/usr/sbin/pptp)
    484k: PID  2369 (/usr/bin/memstat)
   1280k(   1280k): /dev/zero 1755
   1280k(   1280k): /dev/zero 1755
     20k(     16k): /sbin/getty 1568
     64k(     56k): /sbin/rpc.statd 1204
    344k(    316k): /sbin/mdadm 1473
    148k(    124k): /sbin/udevd 467 586 1942 467 586 1942 467 586 1942
     40k(     36k): /sbin/init 1
     24k(     20k): /sbin/portmap 1191
   2136k(     88k): /lib/libgcc_s.so.1 1424
  34932k(     48k): /lib/libnss_files-2.11.3.so 467 586 1204 1222 1424 1473 ...
   2064k(     16k): /lib/libattr.so.1.1.0 1204
  12388k(     76k): /lib/libresolv-2.11.3.so 1222 1545 1606 1755 1948 1956 1...
  12404k(     92k): /lib/libpthread-2.11.3.so 1222 1424 1545 1550 1606 1755 ...
    336k(    120k): /lib/ld-2.11.3.so 1 467 586 1191 1204 1222 1385 1424 147...
  10288k(     28k): /lib/librt-2.11.3.so 1222 1424 1550 1606 1755 1222 1424 ...
  18580k(    112k): /lib/libselinux.so.1 1 467 586 1530 1545 1550 1606 1755 ...
  26704k(     28k): /lib/libnss_compat-2.11.3.so 467 586 1222 1424 1473 1517...
  57108k(   1380k): /lib/libc-2.11.3.so 1 467 586 1191 1204 1222 1385 1424 1...
   4360k(    248k): /lib/libdbus-1.so.3.4.0 1606 1755 1606 1755 1606 1755
  12320k(      8k): /lib/libutil-2.11.3.so 1545 1568 1755 1944 1948 1956 154...
  32916k(     84k): /lib/libnsl-2.11.3.so 467 586 1191 1204 1222 1424 1473 1...
  10292k(     32k): /lib/libcrypt-2.11.3.so 1517 1530 1545 1755 1944 1517 15...
   4124k(     20k): /lib/libnss_dns-2.11.3.so 1948 1956 1948 1956 1948 1956
   2928k(    876k): /lib/libglib-2.0.so.0.2400.2 1606
   4392k(    264k): /lib/libncurses.so.5.7 1493 1760 1493 1760 1493 1760
   2064k(     16k): /lib/libuuid.so.1.3.0 1520
   4108k(     12k): /lib/libcom_err.so.2.1 1545 1755 1545 1755 1545 1755
   2172k(    112k): /lib/libblkid.so.1.1.0 1520
    240k(    236k): /lib/libsepol.so.1 1
  32904k(      8k): /lib/libdl-2.11.3.so 1 467 586 1222 1424 1493 1517 1530 ...
   4624k(    512k): /lib/libm-2.11.3.so 1385 1493 1385 1493 1385 1493
  26716k(     40k): /lib/libnss_nis-2.11.3.so 467 586 1222 1424 1473 1517 15...
   2104k(     52k): /lib/security/pam_unix.so 1755
   2056k(      4k): /lib/security/pam_nologin.so 1755
   2064k(     16k): /lib/security/pam_limits.so 1755
   2052k(      4k): /lib/security/pam_motd.so 1755
   2052k(      4k): /lib/security/pam_deny.so 1755
   2060k(     12k): /lib/security/pam_env.so 1755
   2056k(      8k): /lib/security/pam_mail.so 1755
   2052k(      4k): /lib/security/pam_permit.so 1755
   2064k(     16k): /lib/libcap.so.2.19 1204
   4104k(      8k): /lib/libkeyutils.so.1.3 1545 1755 1545 1755 1545 1755
   2240k(    192k): /lib/libpcre.so.3.12.1 1606
  10292k(     32k): /lib/libwrap.so.0.7.6 1191 1204 1520 1545 1755 1191 1204...
  10288k(     48k): /lib/libpam.so.0.82.2 1517 1530 1545 1755 1944 1517 1530...
   2056k(      8k): /lib/security/pam_ck_connector.so 1755
      8k(      8k): /var/run/pppd2.tdb 1944
    904k(    868k): /bin/bash 1760
    104k(    100k): /bin/dash 1945
   4252k(    148k): /usr/lib/libk5crypto.so.3.1 1545 1755 1545 1755 1545 1755
   4316k(    204k): /usr/lib/libgssapi_krb5.so.2.2 1545 1755 1545 1755
   2208k(    152k): /usr/lib/libexpat.so.1.5.2 1550
    476k(    452k): /usr/sbin/sshd 1545 1755 1545 1755 1545 1755
    108k(    100k): /usr/bin/ssh-agent 1807
  12380k(     92k): /usr/lib/libz.so.1.2.3.4 1424 1493 1545 1606 1755 1807 1...
   4132k(     28k): /usr/lib/libkrb5support.so.0.1 1545 1755 1545 1755
     12k(      8k): /usr/bin/memstat 2369
     44k(     40k): /usr/sbin/acpid 1539
   2188k(    132k): /usr/lib/libpolkit-gobject-1.so.0.0.0 1606
    120k(    112k): /usr/bin/atop 1493
   4936k(    760k): /usr/lib/libkrb5.so.3.3 1545 1755 1545 1755 1545 1755
     40k(     36k): /usr/sbin/cron 1530
     24k(     20k): /usr/sbin/atd 1517
    320k(    296k): /usr/sbin/rsyslogd 1424
   2060k(      8k): /usr/lib/rsyslog/imuxsock.so 1424
   2072k(     24k): /usr/lib/rsyslog/imklog.so 1424
   2068k(     20k): /usr/lib/rsyslog/lmnet.so 1424
   8116k(   1492k): /usr/lib/libcrypto.so.0.9.8 1545 1755 1807 1545 1755 1807
     64k(     60k): /usr/sbin/vnstatd 1385
   2260k(    208k): /usr/lib/libpcap.so.1.1.1 1944
   2060k(     12k): /usr/lib/libck-connector.so.0.0.0 1755
   2756k(    696k): /usr/lib/libgio-2.0.so.0.2400.2 1606
   2064k(     16k): /usr/lib/libgthread-2.0.so.0.2400.2 1606
   2332k(    276k): /usr/lib/libgobject-2.0.so.0.2400.2 1606
   2060k(     12k): /usr/lib/libgmodule-2.0.so.0.2400.2 1606
   2192k(    136k): /usr/lib/libdbus-glib-1.so.2.1.0 1606
   2332k(    276k): /usr/lib/libeggdbus-1.so.0.0.0 1606
    324k(    316k): /usr/bin/dbus-daemon 1550
    136k(    132k): /usr/sbin/console-kit-daemon 1606
     92k(     84k): /usr/sbin/rpc.mountd 1520
    312k(    284k): /usr/sbin/pppd 1944
     72k(     64k): /usr/sbin/pptp 1948 1956 1948 1956
     32k(     20k): /usr/sbin/rpc.idmapd 1222
   2152k(    100k): /usr/lib/libevent-1.4.so.2.1.3 1222
   2076k(     24k): /usr/lib/libnfsidmap.so.0.3.0 1222
     28k(     28k): /usr/lib/gconv/gconv-modules.cache 1530 1606 1760
   2984k(      0k): /usr/lib/locale/locale-archive 1530 1760
   2056k(      8k): /usr/lib/libnfsidmap/nsswitch.so 1222
--------
 633108k (  15844k)

Understanding the output

First, the processes are listed. An amount of memory is shown along with a process ID and the name of the executable which the process is running. The amount of memory shown does not include shared memory: it only includes memory which is private to that process.
After the processes, the shared objects are listed. The amount of memory is shown along with the filename of the shared object, followed by a list of the processes using the
shared object.
Finally, a grand total is shown.

Get data about Apache or Nginx or Lighttpd process only

First, find the pid of web-server:
# pidof httpd # pidof nginx # pidof lighttpd
Use the following syntax to only display data gathered from looking at the process with the PID # 3535
# memstat -p 3535

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin and a trainer for the Linux operating system/Unix shell scripting. He has worked with global clients and in various industries, including IT, education, defense and space research, and the nonprofit sector. Follow him on Twitter, Facebook, Google+.

6 comment

setevoy says:
August 13, 2012 at 6:25 am
Can I see the same in FreeBSD, plz?
Reply Report comment
thedan04 says:
August 15, 2012 at 5:32 am
To combine the two commands presented on one line: memstat -p `pidof httpd`
For FreeBSD you should be able to find memstat with “find /usr/ports/ -name memstat” assuming that your ports tree is found in /usr/ports.
Reply Report comment
1. setevoy says:
  August 15, 2012 at 4:34 pm
  No, nothing:
  19:32:03 [[email protected] /usr/ports] # find /usr/ports/ -name memstat
  19:33:00 [[email protected] /usr/ports] # make search key=memstat
  19:33:17 [[email protected] /usr/ports] # make search name=memstat
  19:33:31 [[email protected] /usr/ports] #
  Reply Report comment
foobrew says:
August 17, 2012 at 11:51 pm
FYI for those people looking, this package doesn’t exist in RHEL. It does exist in fedora but it’s for something completely different (memcached server/cluster). Too bad since that’s a nice tool.
Reply Report comment
nixCraft says:
August 18, 2012 at 12:22 am
As root run the following to build it on RHEL / Fedora / CentOS / SL or any other distro:
```
cd /opt
wget  http://nchc.dl.sourceforge.net/project/memstattool/memstat_0.9.tar.gz
cd memstattool
make
cp memstat.conf /etc/
cp memstat /usr/local/bin/memstat
/usr/local/bin/memstat
```
Note: you may have to adjust PATH settings in /etc/memstat.conf file.
Hope this helps!
Reply Report comment
Anup says:
October 5, 2012 at 11:56 am
Excellent Tool
Reply Report comment

nixCraft