≡ Menu


You can easily start / stop / pause or take a snapshot from a shell prompt under a Linux / Windows host using vmrun command. This is useful if you do not want to run web interface for starting and/or stopping VMs.

vmrun commands

vmrun -u USER -h 'https://vmware.server.com:8333/sdk' -p PASSWORD COMMAND [PARAMETERS]
vmrun -u USER -h 'https://vmware.server.com:8333/sdk' -p PASSWORD start "[storage] Path/to/.vmx"

=> -u USER : VMWare server username

=> -h 'https://vmware.server.com:8333/sdk' : Local or remote server FQDN / IP address

=> -p PASSWORD : VMWare server password

=> COMMAND [PARAMETERS] : Command can be any one of the following:

--------------           ----------           -----------
start                    Path to vmx file     Start a VM
stop                     Path to vmx file     Stop a VM
reset                    Path to vmx file     Reset a VM
suspend                  Path to vmx file     Suspend a VM
pause                    Path to vmx file     Pause a VM
unpause                  Path to vmx file     Unpause a VM

Start a VM called CentOS

To start a virtual machine with Vmware server 2.0 on a Linux host, stored on storage called iscsi:
vmrun -T server -h 'https://vms.nixcraft.in:8333/sdk' -u root -p 'secrete' start "[iSCSI] CentOS52_64/CentOS52_64.vmx"
To start a virtual machine with Workstation on a Windows host (open command prompt by visiting Start > Run > cmd > [enter] key):
vmrun -T ws start "c:\My VMs\centos\centos.vmx"

Stop a VM called CentOS

To stop a virtual machine with Vmware server 2.0 on a Linux host, stored on storage called iscsi:
vmrun -T server -h 'https://vms.nixcraft.in:8333/sdk' -u root -p 'secrete' stop "[iSCSI] CentOS52_64/CentOS52_64.vmx"

Reset a VM called Debian

To reset a virtual machine with Vmware server 2.0 on a Linux host, stored on storage called DISK3:
vmrun -T server -h 'https://sun4k.nixcraft.co.in:8333/sdk' -u root -p 'secrete' reset "[DISK3] Debian5/Debian5.vmx"

A Redundant Array of Independent Drives (or Disks), also known as Redundant Array of Inexpensive Drives (or Disks) (RAID) is an term for data storage schemes that divide and/or replicate data among multiple hard drives. RAID can be designed to provide increased data reliability or increased I/O performance, though one goal may compromise the other. There are 10 RAID level. But which one is recommended for data safety and performance considering that hard drives are commodity priced?

I did some research in last few months and based upon my experince I started to use RAID10 for both Vmware / XEN Virtualization and database servers. A few MS-Exchange and Oracle admins also recommended RAID 10 for both safety and performance over RAID 5.

Quick RAID 10 overview (raid 10 explained)

RAID 10 = Combining features of RAID 0 + RAID 1. It provides optimization for fault tolerance.

RAID 0 helps to increase performance by striping volume data across multiple disk drives.

RAID 1 provides disk mirroring which duplicates your data.

In some cases, RAID 10 offers faster data reads and writes than RAID 5 because it does not need to manage parity.

Fig.01: Raid 10 in action

Fig.01: Raid 10 in action

RAID 5 vs RAID 10

From Art S. Kagel research findings:

If a drive costs $1000US (and most are far less expensive than that) then switching from a 4 pair RAID10 array to a 5 drive RAID5 array will save 3 drives or $3000US. What is the cost of overtime, wear and tear on the technicians, DBAs, managers, and customers of even a recovery scare? What is the cost of reduced performance and possibly reduced customer satisfaction? Finally what is the cost of lost business if data is unrecoverable? I maintain that the drives are FAR cheaper! Hence my mantra:

Is RAID 5 Really a Bargain?

Cary Millsap, manager of Hotsos LLC and the editor of Hotsos Journal found the following facts - Is RAID 5 Really a Bargain?":

  • RAID 5 costs more for write-intensive applications than RAID 1.
  • RAID 5 is less outage resilient than RAID 1.
  • RAID 5 suffers massive performance degradation during partial outage.
  • RAID 5 is less architecturally flexible than RAID 1.
  • Correcting RAID 5 performance problems can be very expensive.

My practical experience with RAID arrays configuration

To make picture clear, I'm putting RAID 10 vs RAID 5 configuration for high-load database, Vmware / Xen servers, mail servers, MS - Exchange mail server etc:

RAID LevelTotal array capacityFault toleranceRead speedWrite speed
500GB x 4 disks
1000 GB1 disk4X2X
500GB x 3 disks
1000 GB1 disk2XSpeed of a RAID 5 depends upon the controller implementation

You can clearly see RAID 10 outperforms RAID 5 at fraction of cost in terms of read and write operations.

A note about backup

Any RAID level will not protect you from multiple disk failures. While one disk is off line for any reason, your disk array is not fully redundant. Therefore, old good tape backups are always recommended.

Please add your thoughts and experience in the comments below.

Further readings:

Seagate Barracuda: 1.5TB Hard Drive Launched

Wow, this is a large size desktop hard disk for storing movies, tv shows, music / mp3s, and photos. You can also load multiple operating systems using vmware or other software for testing purpose. This hard disk comes with 5 year warranty and can transfer at 300MB/s. From the article:

It's been more than 18 months since Hitachi reached the terabyte mark with the Deskstar 7K1000. In that time, all the major players in the hard drive industry have spun up terabytes of their own, and in some cases, offered multiple models targeting different markets. With so many options available and more than enough time for the milestone capacity's initial buzz to fade, it's no wonder that the current crop of 1TB drives is more affordable than we've ever seen from a flagship capacity. The terabyte, it seems, is old news.

Fig.01: Seagate's Barracuda 7200.11 1.5TB hard drive

Fig.01: Seagate's Barracuda 7200.11 1.5TB hard drive

The real question is about reliability. How reliable is the hard disk? So far my Seagate 500GB hard disk working fine. I might get one to dump all my multimedia data / files :)

A few days ago I noticed that NFS performance between a web server node and NFS server went down by 50%. NFS was optimized and the only thing was updated Red Hat kernel v5.2. I also noticed same trend on CentOS 5.2 64 bit edition.

NFS server crashed each and every time web server node tried to store a large file 20-100 MB each. Read performance was fine but write performance went to hell. Finally, I had to rollback the updates. Recently, while reading Red Hat site I came across the solution.

Updated kernel packages that fix various security issues and several bugs are now available for Red Hat Enterprise Linux 5:

* a 50-75% drop in NFS server rewrite performance, compared to Red Hat
Enterprise Linux 4.6, has been resolved.

After upgrading kernel on both server and client my issue resolved:
# yum update

Following are few situations where you may be interested in performing a filesystem benchmarking.

=> Deploying a new application that is very read and write intensive.
=> Purchased a new storage system and would like to measure the performance.
=> Changing the RAID level and would like to measure the performance of the new RAID.
=> Changing the storage parameters and would like to know the performance impact of this change

This article gives you a jumpstart on performing benchmark on filesystem using iozone a free Filesystem Benchmark utility.

1. Download and Install iozone software

Go to iozone and download the iozone for your appropriate platform. I downloaded the "Linux i386 RPM". Install the iozone from the RPM as shown below. By default this installs the iozone under /opt/iozone/bin
# cd /tmp
# wget http://www.iozone.org/src/current/iozone-3-303.i386.rpm
# rpm -ivh iozone-3-303.i386.rpm

Sample output:

Preparing...                ########################################### [100%]
   1:iozone                 ########################################### [100%]

Note: You can install iozone under any UNIX / Linux or Windows operating system.

2. Start the performance test

Execute the following command in the background to begin the performance test.

# /opt/iozone/bin/iozone -R -l 5 -u 5 -r 4k -s 100m -F /home/f1 /home/f2 /home/f3 /home/f4 /home/f5 | tee -a /tmp/iozone_results.txt &

Let us review all the individual parameter passed to iozone command.

  • -R : Instructs the iozone to generate excel compatible text output.
  • -l : This is the lower limit on how many process/threads needs to be started by iozone during execution. In this example, iozone will start 5 threads.
  • -u : This is the upper limit on how many process/threads needs to be started by iozone during execution. In this example, iozone will not exceed maximum of 5 threads. If you set -l and -u to the same value, it will run exactly those many number of process/threads. In this example, this will execute exactly 5 threads.
  • -r : This specifies the record size. In this example, the record size for benchmark testing is 4k. This is an important parameter to be set appropriately depending on the purpose of your filesystem performance testing. For e.g. If you are performing benchmark on a filesystem that will host a database, it is appropriate to set this value to the DB block size of the database.
  • -s : This specifies the size of the file that needs to be tested. In this example, iozone will try to perform test on 100Mb file.
  • -F : Specify the temporary filename that should be used by the iozone during testing. The total number of files specified here should match the value specified in -l and -u parameter.

3. Analyze the output of iozone file.

The first part of the output will contain the details about every individual filesystem performance metrics that was tested. for e.g. Initial write, rewrite etc as shown below.

        Iozone: Performance Test of File I/O
                Version $Revision: 3.303 $
                Compiled for 32 bit mode.
                Build: linux
        Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
                     Al Slater, Scott Rhine, Mike Wisner, Ken Goss
                     Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
                     Randy Dunlap, Mark Montague, Dan Million,
                     Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
                     Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root.
        Run began: Thu Jun  22 00:08:51 2008
        Excel chart generation enabled
        Record Size 4 KB
        File size set to 102400 KB
        Command line used: /opt/iozone/bin/iozone -R -l 5 -u 5 -r  4k -s 100m -F /home/f1 /home/f2 /home/f3 /home/f4 /home/f5
        Output is in Kbytes/sec
        Time Resolution = 0.000001 seconds.
        Processor cache size set to 1024 Kbytes.
        Processor cache line size set to 32 bytes.
        File stride size set to 17 * record size.
        Min process = 5
        Max process = 5
        Throughput test with 5 processes
        Each process writes a 102400 Kbyte file in 4 Kbyte records
        Children see throughput for  2 initial writers  =   60172.28 KB/sec
        Parent sees throughput for  2 initial writers   =   45902.89 KB/sec
        Min throughput per process                        =   28564.52 KB/sec
        Max throughput per process                      =   31607.76 KB/sec
        Avg throughput per process                      =   30086.14 KB/sec
        Min xfer                                                  =   92540.00 KB
        Children see throughput for  2 rewriters        =   78658.92 KB/sec
        Parent sees throughput for  2 rewriters         =   34277.52 KB/sec
        Min throughput per process                      =   35743.92 KB/sec
        Max throughput per process                      =   42915.00 KB/sec
        Avg throughput per process                      =   39329.46 KB/sec
        Min xfer                                        =   85296.00 KB

Similar values like above will be generated for readers, re-readers, reverse readers, stride readers, random readers, mixed workload, random writers, pwrite writers, pread readers. The last part of the iozone output will contain the Throughput summary for different metrics as shown below.

Throughput report Y-axis is type of test X-axis is number of processes
Record size = 4 Kbytes
Output is in Kbytes/sec
Initial write       60172.28
Rewrite           78658.92
Read              2125613.88
Re-read          1754367.31
Reverse Read 1603521.50
Stride read      1633166.38
Random read   1583648.75
Mixed workload 1171437.78
Random write    5365.59
Pwrite               26847.44
Pread               2054149.00

(Fig.01: iozone in action)

Iozone does a benchmarking on different types of file system performance metrics. for e.g. Read, Write, Random read. Depending on the application that you are planning to deploy on that particular filesystem, pay attention to the appropriate items. for e.g. If the filesystem hosts an read intensive OLTP database, then pay attention to Random Read, Random write and Mixed workload. If the application does lot of streaming media content, pay attention to the Sequential Read. On a final note, you can generate graphs using the Generate_Graphs and gengnuplot.sh located under /opt/iozone/bin, based on the iozone output.


  • Iozone PDF documentation – Full documentation from iozone.org explaining all the iozone command line options and more.
  • Linux Iozone graph example - This is a sample *.xls file from iozone that shows the kind of excel output that can be generated from iozone.

Good news and great contribution from HP. You can study all those advanced features for academic project.

AdvFS is a file system that was developed by Digital Equipment Corp and continues to be part of HP's
Tru64 operating system. It offers many advanced features. Continuing its efforts to advance customer adoption of Linux, HP today announced the contribution of its Tru64 UNIX Advanced File System (AdvFS) source code to the open source community. The code on this site is licensed under the GPLv2 to be compatible with the Linux kernel.

The AdvFS source code includes capabilities that increase uptime, enhance security and help ensure maximum performance of Linux file systems. HP will contribute the code as a reference implementation of an enterprise Linux file system under the terms of General Public License Version 2 for compatibility with the Linux kernel, as well as provide design documentation, test suites and engineering resources.

Now the million dollar question - Is there any reason to pick AdvFS fs over any of the other 20+ file systems such as XFS/ext2/ext3 under Linux?

A sudden outburst of violent disk I/O activity can bring down your email or web server. Usually, a web / mysql or mail server serving millions and millions pages per months are prone to this kind of problem. Backup activity can increase current system load. To avoid this kind of sudden outburst problem, run your script with scheduling class and priority. Linux comes with various utilities to manage this kind of madness.
[click to continue…]