How To Create sar Graphs With kSar To Identifying Linux Bottlenecks

The sar command collects, report, or save UNIX / Linux system activity information. It will save selected counters in the operating system to the /var/log/sa/sadd file. From the collected data, you get lots of information about your server:

  1. CPU utilization
  2. Memory paging and its utilization
  3. Network I/O, and transfer statistics
  4. Process creation activity
  5. All block devices activity
  6. Interrupts/sec etc.

The sar command output can be used for identifying server bottlenecks. However, analyzing information provided by sar can be difficult, so use kSar tool. kSar takes sar command output and plots a nice easy to understand graph over a period of time.

sysstat Package

The sar, sa1, and sa2 commands are part of sysstat package. Collection of performance monitoring tools for Linux includes

  1. sar : Displays the data.
  2. sa1 and sa2: Collect and store the data for later analysis. The sa2 shell script write a daily report in the /var/log/sa directory. The sa1 shell script collect and store binary data in the system activity daily data file.
  3. sadc – System activity data collector. You can configure various options by modifying sa1 and sa2 scripts. They are located at the following location:
    • /usr/lib64/sa/sa1 (64bit) or /usr/lib/sa/sa1 (32bit) – This calls sadc to log reports to/var/log/sa/sadX format.
    • /usr/lib64/sa/sa2 (64bit) or /usr/lib/sa/sa2 (32bit) – This calls sar to log reports to /var/log/sa/sarX format.

How do I install sar on my system?

Type the following yum command to install sysstat on a CentOS/RHEL based system:
# yum install sysstat
Sample outputs:

Loaded plugins: downloadonly, fastestmirror, priorities,
              : protectbase, security
Loading mirror speeds from cached hostfile
 * addons:
 * base:
 * epel:
 * extras:
 * updates:
0 packages excluded due to repository protections
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package sysstat.x86_64 0:7.0.2-3.el5 set to be updated
--> Finished Dependency Resolution
Dependencies Resolved
 Package        Arch          Version             Repository   Size
 sysstat        x86_64        7.0.2-3.el5         base        173 k
Transaction Summary
Install      1 Package(s)
Update       0 Package(s)
Remove       0 Package(s)
Total download size: 173 k
Is this ok [y/N]: y
Downloading Packages:
sysstat-7.0.2-3.el5.x86_64.rpm               | 173 kB     00:00
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing     : sysstat                                      1/1
  sysstat.x86_64 0:7.0.2-3.el5

Configuration files for sysstat

Edit /etc/sysconfig/sysstat file specify how long to keep log files in days, maximum is a month:
# vi /etc/sysconfig/sysstat
Sample outputs:

# keep log for 28 days
# the default is 7

Save and close the file.

Find the default cron job for sar

The default cron job is located at /etc/cron.d/sysstat:
# cat /etc/cron.d/sysstat
Sample outputs:

# run system activity accounting tool every 10 minutes
*/10 * * * * root /usr/lib64/sa/sa1 1 1
# generate a daily summary of process accounting at 23:53
53 23 * * * root /usr/lib64/sa/sa2 -A

Tell sadc to report statistics for disks

Edit the /etc/cron.d/sysstat file using a text editor such as NA command or vim command, enter:
# vi /etc/cron.d/sysstat
Update it as follows to log all disk stats (the -d option force to log stats for each block device and the -I option force report statistics for all system interrupts):

# run system activity accounting tool every 10 minutes
*/10 * * * * root /usr/lib64/sa/sa1 -I -d 1 1
# generate a daily summary of process accounting at 23:53
53 23 * * * root /usr/lib64/sa/sa2 -A 

On a CentOS/RHEL 7.x you need to pass the -S DISK option to collect data for block devices. Pass the -S XALL to collect data about:

  1. Disk
  2. Partition
  3. System interrupts
  4. SNMP
  5. IPv6
# Run system activity accounting tool every 10 minutes
*/10 * * * * root /usr/lib64/sa/sa1 -S DISK 1 1
# 0 * * * * root /usr/lib64/sa/sa1 600 6 &
# Generate a daily summary of process accounting at 23:53
53 23 * * * root /usr/lib64/sa/sa2 -A

Save and close the file. Turn on the service for a CentOS/RHEL version 5.x/6.x, enter:
# chkconfig sysstat on
# service sysstat start

Sample outputs:

Calling the system activity data collector (sadc): 

For a CentOS/RHEL 7.x, run the following commands:
# systemctl enable sysstat
# systemctl start sysstat.service
# systemctl status sysstat.service

Sample outputs:

● sysstat.service - Resets System Activity Logs
   Loaded: loaded (/usr/lib/systemd/system/sysstat.service; enabled; vendor preset: enabled)
   Active: active (exited) since Sat 2018-01-06 16:33:19 IST; 3s ago
  Process: 28297 ExecStart=/usr/lib64/sa/sa1 --boot (code=exited, status=0/SUCCESS)
 Main PID: 28297 (code=exited, status=0/SUCCESS)
Jan 06 16:33:19 centos7-box systemd[1]: Starting Resets System Activity Logs...
Jan 06 16:33:19 centos7-box systemd[1]: Started Resets System Activity Logs.

How Do I Use sar? How do I View Stats?

Use the sar command to display output the contents of selected cumulative activity counters in the operating system. In this example, sar is run to get real-time reporting from the command line about CPU utilization:
# sar -u 3 10
Sample outputs:

Linux 2.6.18-164.2.1.el5 ( 	12/14/2009

09:49:47 PM       CPU     %user     %nice   %system   %iowait    %steal     %idle
09:49:50 PM       all      5.66      0.00      1.22      0.04      0.00     93.08
09:49:53 PM       all     12.29      0.00      1.93      0.04      0.00     85.74
09:49:56 PM       all      9.30      0.00      1.61      0.00      0.00     89.10
09:49:59 PM       all     10.86      0.00      1.51      0.04      0.00     87.58
09:50:02 PM       all     14.21      0.00      3.27      0.04      0.00     82.47
09:50:05 PM       all     13.98      0.00      4.04      0.04      0.00     81.93
09:50:08 PM       all      6.60      6.89      1.26      0.00      0.00     85.25
09:50:11 PM       all      7.25      0.00      1.55      0.04      0.00     91.15
09:50:14 PM       all      6.61      0.00      1.09      0.00      0.00     92.31
09:50:17 PM       all      5.71      0.00      0.96      0.00      0.00     93.33
Average:          all      9.24      0.69      1.84      0.03      0.00     88.20


  • 3 = interval
  • 10 = count

To view process creation statistics, enter:
# sar -c 3 10
To view I/O and transfer rate statistics, enter:
# sar -b 3 10
To view paging statistics, enter:
# sar -B 3 10
To view block device statistics, enter:
# sar -d 3 10
To view statistics for all interrupt statistics, enter:
# sar -I XALL 3 10
To view device specific network statistics, enter:
# sar -n DEV 3 10
# sar -n EDEV 3 10

To view CPU specific statistics, enter:
# sar -P ALL
# Only 1st CPU stats
# sar -P 1 3 10

To view queue length and load averages statistics, enter:
# sar -q 3 10
To view memory and swap space utilization statistics, enter:
# sar -r 3 10
# sar -R 3 10

To view status of inode, file and other kernel tables statistics, enter:
# sar -v 3 10
To view system switching activity statistics, enter:
# sar -w 3 10
To view swapping statistics, enter:
# sar -W 3 10
To view statistics for a given process called Apache with PID # 3256, enter:
# sar -x 3256 3 10

Say Hello To kSar

sar and sadf provides CLI based output. The output may confuse all new users / sys admin. So you need to use kSar which is a java application that graph your sar data. It also permit to export data to PDF/JPG/PNG/CSV. You can load data from three method : local file, local command execution, and remote command execution via SSH. kSar supports the sar output of the following OS:

  1. Solaris 8, 9 and 10
  2. Mac OS/X 10.4+
  3. Linux (Systat Version >= 5.0.5)
  4. AIX (4.3 & 5.3)
  5. HPUX 11.00+

Download And Install kSar

Visit the official website and grab the latest source code. Use wget to download the source code, enter:
$ wget

How Do I Run kSar?

Make sure JAVA jdk is installed and working correctly. Type the following command to start kSar, run:
$ java -jar ksar-5.2.4-SNAPSHOT-all.jar

Fig.01: kSar welcome screen

Fig.01: kSar welcome screen

Next you will see main kSar window, and menus with two panels.
Fig.02: kSar - the main window

Fig.02: kSar - the main window

The left one will have a list of graphs available depending on the data kSar has parsed. The right window will show you the graph you have selected.

How Do I Generate sar Graphs Using kSar?

First, you need to grab sar command statistics from the server named server1. Type the following command to get stats, run:
[server1 ]# LC_ALL=C sar -A > /tmp/
Next copy file to local desktop from a remote box using the scp command:
[desktop ]$ scp /tmp/
Switch to kSar Windows. Click on Data > Load data from text file > Select from /tmp/ > Click the Open button.
Now, the graph type tree is deployed in left pane and a graph has been selected:

Fig.03: Processes for server1

Fig.03: Processes for server1

Fig.03: Disk stats (blok device) stats for server1

Fig.04: Disk stats (blok device) stats for server1

Fig.05: Memory stats for server1

Fig.05: Memory stats for server1

Zoom in and out

Using the move, you can interactively zoom onto up a part of a graph. To select a zone to zoom, click on the upper left conner and while still holding the mouse but on move to the lower-right of the zone you want to zoom. To come back to unzoomed view click and drag the mouse to any corner location except a lower-right one. You can also right click and select zoom options

Understanding kSar Graphs And sar Data

I strongly recommend reading sar and sadf command man page:
$ man sar
$ man sadf

Case Study: Identifying Linux Server CPU Bottlenecks

With sar command and kSar tool, one can get the detailed snapshot of memory, CPU, and other subsystems. For example, if CPU utilization is more than 80% for a long period, a CPU bottleneck is most likely occurring. Using sar -x ALL you can find out CPU eating process. The output of mpstat command (part of sysstat package itself) will also help you understand the cpu utilization. You can easily analyze this information with kSar.

I Found CPU Bottlenecks…

Performance tuning options for the CPU are as follows:

  1. Make sure that no unnecessary programs are running in the background. Turn off all unnecessary services on Linux.
  2. Use cron to schedule jobs (e.g., backup) to run at off-peak hours.
  3. Use top and ps command to find out all non-critical background jobs / services. Make sure you lower their priority using renice command.
  4. Use taskset command to set a processes’s CPU affinity (offload cpu) i.e. bind processes to different CPUs. For example, run MySQL database on cpu #2 and Apache on cpu # 3.
  5. Make sure you are using latest drivers and firmware for your server.
  6. If possible add additional CPUs to the system.
  7. Use faster CPUs for a single-threaded application (e.g. Lighttpd web server app).
  8. Use more CPUs for a multi-threaded application (e.g. MySQL database server app).
  9. Use more computer nodes and set up a load balancer for a web app.

isag – Interactive System Activity Grapher (alternate tool)

The isag command graphically displays the system activity data stored in a binary data file by a previous sar run. The isag command invokes sar to extract the data to be plotted. isag has limited set of options as compare to kSar.

Fig.06: isag CPU utilization graphs

Fig.06: isag CPU utilization graphs


🐧 Get the latest tutorials on Linux, Open Source & DevOps via RSS feed or Weekly email newsletter.

🐧 41 comments so far... add one
CategoryList of Unix and Linux commands
Disk space analyzersncdu pydf
File Managementcat
FirewallAlpine Awall CentOS 8 OpenSUSE RHEL 8 Ubuntu 16.04 Ubuntu 18.04 Ubuntu 20.04
Network UtilitiesNetHogs dig host ip nmap
OpenVPNCentOS 7 CentOS 8 Debian 10 Debian 8/9 Ubuntu 18.04 Ubuntu 20.04
Package Managerapk apt
Processes Managementbg chroot cron disown fg jobs killall kill pidof pstree pwdx time
Searchinggrep whereis which
User Informationgroups id lastcomm last lid/libuser-lid logname members users whoami who w
WireGuard VPNAlpine CentOS 8 Debian 10 Firewall Ubuntu 20.04
41 comments… add one
  • Bhavya Dec 15, 2009 @ 10:00

    I think u’ve given the wrong path to the images (screenshots of ksar), They are not being loaded, please check & update.


  • 🐧 nixCraft Dec 15, 2009 @ 10:31


    They are loading fine here. I do not see any problem.


    --2009-12-15 16:00:35--
    Connecting to||:80... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 100548 (98K) [image/png]
    Saving to: `memory-ksar.png'
    100%[====================================================================================================================>] 1,00,548    68.5K/s   in 1.4s
    2009-12-15 16:00:37 (68.5 KB/s) - `memory-ksar.png' saved [100548/100548]
  • Jaysunn Dec 15, 2009 @ 13:38

    I have used KSAR many times to provide quality graphs for management. What A great tutorial.



  • Mark Stoecker Dec 15, 2009 @ 20:21

    Thank you very much for the post. I enjoy reading your blog. I am a MySQL DBA and this will help me tremendously in my job. Thanks again!!

  • sims Dec 16, 2009 @ 0:21

    Very very very interesting. I haven’t heard of sar or ksar. Good info! Thanks!

  • 🐧 nixCraft Dec 16, 2009 @ 6:04


    Yes those graphs makes more sense with management folks.

    @mark / sims – thanks for comment.

  • Mario Vanoni Dec 16, 2009 @ 14:15

    very old version, see
    Wednesday November 11th, 2009
    Sysstat 9.0.6 released (stable version).
    feedback only

  • 🐧 nixCraft Dec 16, 2009 @ 15:04


    Thanks for the feedback. However, RHEL / CentOS got older stable version. I already found some good stuff in latest version.

  • Tapas Mallick Dec 17, 2009 @ 6:30

    “Download PDF” lisk is not working, showing “410 Gone”.

  • 🐧 nixCraft Dec 17, 2009 @ 17:03


    I’ve tweaked code a little bit. Try again. This time it should work!

  • Dec 18, 2009 @ 17:32
  • Liju Dec 19, 2009 @ 4:37

    Great info !!!!!

    Really help me lots to get a portable output. Earlier I was used cacti to do this. It’s a online tool and required huge SNMP traffic to analyze. But it’s simple, easy to configure but rich in output.


  • Jalal Hajigholamali Jan 11, 2010 @ 4:00

    Useful package, thanks a lot

  • nima0102 Feb 15, 2010 @ 15:03

    First of all thanks for your good article.
    I am using munin as monitoring system resources and I think you try it.


  • Tarique Feb 16, 2010 @ 11:01

    Useful link!! Thanks

  • john Jun 22, 2010 @ 15:26

    What if you did not generate SAR file using the locale LC_ALL=C prefix?

    I have a sar output in text (without using the locale prefix), but kSar doesn’t parse it. It only knows first timestamp and that’s it.

    Any options? Can I tweak the file to see the output?

  • Kester RIley Jan 19, 2011 @ 10:03

    Why when I did “sar -A 5 0” yesterday did it work fine

    today after a nw install on a new machine it always opens the file since midnight, and does not update every 5 seconds. Infact nothing I do can make it update “Live” it always opens from midnight to now????

  • tangosara1 Sep 16, 2011 @ 21:12

    Excellent information and very nicely explained. Much appreciate all the effort in doing so. Cheers, Tangosara1

  • Jacobo Sep 21, 2011 @ 20:12

    Hi, I took the liberty to create a script to get the sar statistics, please feel free to modify it if you like it

    Thank you.


    • Peter Kerekes Mar 28, 2014 @ 22:11


  • Bijohn Vincent Sep 23, 2011 @ 4:51

    Great !!!!!!!!!!!!! Very useful article….

    But there is some issues with LC_ALL=C sar -A > 1234.txt.
    The -A option………….. It is throwing java exceptions.

    java.lang.NumberFormatException: For input string: “kbhugfree”
    at sun.misc.FloatingDecimal.readJavaFormatString(
    at java.lang.Float.valueOf(
    at java.lang.Float.(
    at net.atomique.ksar.Linux.Parser.parse(
    at net.atomique.ksar.kSar.parse(

    If I use ,
    bijohn@lucid:~$ LC_ALL=C sar -H > 1234.txt
    It is not generating java exceptions. But not plotting output.

    Other options are Working fine :
    bijohn@lucid:~$ LC_ALL=C sar -bqr > 1234.txt

    Any suggestion ????

    • Peter Sep 13, 2012 @ 12:10


      To avoid java.lang.NumberFormatException set the following :

      export LC_ALL=C
      export S_TIME_FORMAT=ISO

      Before using sar or ksar.


  • sid Nov 21, 2011 @ 22:59

    One of the biggest flaws with this tool is that we need to run our servers using X session which is not a good idea.

    Is there a way to get the reports through web pages using Apache or pull reports off to another server and do the same thing ?

  • Vishal Singh Mar 3, 2012 @ 5:33

    Great !!!!!!!!!!!!! Very useful article….
    Thanks Team cyberciti

    You Guys are great


  • Edray Mashiri Apr 18, 2012 @ 9:06

    Hi Guys

    Is there a way to generate weekly or monthly reports with this tool?


  • Cristian Alejandro Aug 10, 2012 @ 7:23

    Hi!, great article. You should update the link

  • Chuong Aug 30, 2012 @ 8:18

    Dear Vivek,

    I am very interesting your post. I tried it. But i have 1 question. On the sysstat config file the system stats are kept in 28 days. So, how can i do if i want to see the history system stats (not currently). Such as, the system stats of the first day.

    So thanks and Best regards

  • Steve Ritenour Oct 23, 2012 @ 19:56

    I too am very interested in doing weekly and/or monthly reports. Is that capability available in ksar? I haven’t been able to find it.


  • Josh Feather Nov 12, 2012 @ 20:13

    Nice tool. Runs in batch mode for easy daily/weekly/monthly reporting. Only flaw is that when run in batch mode, it only displays up to 18:00 – using either jpg, png or pdf. Any ideas?

  • Luis Calvo Nov 28, 2012 @ 14:43

    Hi, it is a great article !

    It has been very usefull in my job.

    Thank’s a lot.

  • danny Dec 7, 2012 @ 9:39

    Guys, can somebody help me to extract pdf for CPU and Memory in PDF form, remaining graphs are just junk for me. The following line i used to create a PDF file.
    java -jar /kSar.jar -input -outputPDF output.pdf.


    • Arunabh Feb 6, 2017 @ 5:36

      Hi Danny,

      I use ksar version 5.0.6, where you’ll get an option to export the report to PDF, JPG, PNG, CSV, & XLS format. Once you select PDF option and the location to save it, it’ll give you choice to import all data or selected options (tick the check box you want i.e. CPU & memory etc).

  • Manish Jan 5, 2013 @ 18:00

    What a good article, i was aware of sar but using different tools like nmon and bonniee++ to create graphs, but ksar is really wonderful and easy to use.

    thanks again

  • harsha Feb 28, 2013 @ 11:11

    sar -x option never works.
    I am currently using SUSE 11 Enterprise Linux

    When I run the SAR command using “-x” options it says that option does not exist. I cant find “-x” even in man page for sar.
    Can you please help ??

    I really need a proper “sar ” command to monitor activity for Given process ( say Apache PID 3618) … please ????

    • samir parekh May 10, 2013 @ 10:51

      sar -x

  • Charles Stepp Oct 7, 2014 @ 17:21

    In the crontab entry, you should not be limiting the interval to 1 second. Sar uses the same system resources no matter how long the interval is. It reads kernel values, sleeps, reads the values again and records/prints the difference value. 1 second, 10 seconds, 1200 seconds are the same as far as sar’s resource usage. 99.99% of sar’s usage is sleep, which is what the kernel does anyway when it’s not doing anything. Note below that the first sar sample of only a second showed an average cpu of 3%. The longer samples, averaging over a longer period, show that 6% is probably more of an accurate average, at this time. The web pages I’ve seen so far feed each other with this 1 second sample thing, almost like someone is afraid sar might bog the system down. It won’t. The same two sets of kernel reads happens no matter what the interval is:
    time sar 1 1; time sar 10 1; time sar 100 1
    Linux 2.6.18-194.el5 (blahblah) 10/07/14
    12:04:51 CPU %user %nice %system %iowait %steal %idle
    12:04:52 all 3.00 0.00 0.75 0.00 0.00 96.25
    Average: all 3.00 0.00 0.75 0.00 0.00 96.25
    sar 1 1 0.00s user 0.00s system 0% cpu 1.005 total
    Linux 2.6.18-194.el5 (blahblah) 10/07/14
    12:04:52 CPU %user %nice %system %iowait %steal %idle
    12:05:02 all 6.21 0.00 0.93 0.20 0.00 92.67
    Average: all 6.21 0.00 0.93 0.20 0.00 92.67
    sar 10 1 0.00s user 0.00s system 0% cpu 10.005 total
    Linux 2.6.18-194.el5 (blahblah) 10/07/14
    12:05:02 CPU %user %nice %system %iowait %steal %idle
    12:06:42 all 6.32 0.00 0.97 0.24 0.00 92.47
    Average: all 6.32 0.00 0.97 0.24 0.00 92.47
    sar 100 1 0.00s user 0.00s system 0% cpu 1:40.01 total
    From the man page example it shows each hour having 3 20 minute samples. This provides accurate averaging and small sa## files. A 1 second interval each 10 minutes is 1/600th of the information available.
    To create a daily record of sar activities, place the following entry
    in your root or adm crontab file:
    0 8-18 * * 1-5 /usr/lib/sa/sa1 1200 3 &

  • Sonu Mar 18, 2015 @ 0:50

    Thanks for the nice tutorial.
    Can you please share the way to get stats for multiple days in a single file so that it can be imported in ksar for analysis

  • Din Mar 20, 2015 @ 10:24

    Its a nice tool. I have few questions
    1. How to generate the monthly or weekly report ? i mean in a single chart i should a get a monthly report ?
    2. -addHTML option is not working. I am not able to generate any HTML report.

    Anyone help me


  • susie Oct 1, 2015 @ 18:10

    How do you get a single graph for 10 days worth of data for CPU, I/o and memory usages.
    Appreciate your help.


  • dead Apr 20, 2016 @ 19:13

    Dead link

  • jose Jul 25, 2017 @ 18:22

    necesito un sar que este en un rango de un mes anterior help

Leave a Reply

Your email address will not be published.

Use HTML <pre>...</pre> for code samples. Still have questions? Post it on our forum