≡ Menu


This is an interesting filesystem comparison. If you are looking to build cheap storage for personal use file system decision is quite important:

This is my attempt to cut through the hype and uncertainty to find a storage subsystem that works. I compared XFS and EXT4 under Linux with ZFS under OpenSolaris. Aside from the different kernels and filesystems, I tested internal and external journal devices and software and hardware RAIDs. Software RAIDs are "raid-10 near2" with 6 disks on Linux. On Solaris the zpool is created with three mirrors of two disks each. Hardware RAIDs use the Areca's RAID-10 for both Linux and Solaris. Drive caches are disabled throughout, but the battery-backed cache on the controller is enabled when using hardware RAID.

=> ZFS, XFS, and EXT4 filesystems compared

Why my Linux server ext3 filesystem go read-only?

From my mailbag:

We have 5 Dell server collocated running CentOS 4.x and 5.x server operating system. Sometime my file system (ext3) goes read-only. I’d like to know what could be causing such a problem?

My guess:
a) Hardware problem / hard disk problem, check harddisk for errors.

b) High disk I/O aka busy I/O retry error can mark low level disk call as failed. This will force ext3 to go into read only mode.

c) High disk I/O on SAN

d) SAN is not configured properly for the path failover.

In all sort of problems ext3 goes read-only to protect the filesystem and further damage. If you are using VMWARE, check out official webpage to download SCSI patches or workaround for vmware problems.

So what could be causing the file system on Linux go read-only?

Apart from above generic problem, any other error can trigger filesystem on Linux go read only. I hope our reader / seasoned Linux admin can help to answer this question. Please share the experiences and advice in the comments.

ZFS on Linux using FUSE

Linus Torvalds called ZFS one of the very few bright spots in Solaris. ZFS released under Sun's Common Development and Distribution License. Linux is under GPL v2 which makes ZFS incompatible with Linux. I’ve used ZFS under Solaris and it is absolutely rocking file system.

Now this hot feature from OpenSolaris is taking a weird but working path to Linux inclusion using FUSE (Filesystem in Userspace).

If you like Linux for the long list of supported hardware but Solaris for the advanced new ZFS filesystem, a new development project might have a happy surprise for you. One advantage to putting ZFS into a separate daemon is the same as something that microkernel operating system developers have been talking up for years: You can kill and restart the filesystem independently of the operating system.

Check out official ZFS on FUSE blog and LinuxWorld article for more information ZFS on Linux: It's alive

Configure lighttpd alias (mod_alias)

This lighttpd module provides for mapping different parts of the host filesystem in the document tree. You can use it for mapping various directories. For example cgi-bin directory mapped to /var/lib/cgi-bin. The alias module is used to specify a special document-root for a given url-subset.


Open your lighttpd configuration file:
vi /etc/lighttpd/lighttpd.conf

Append/add mod_ alias to list of server modules:
server.modules += ( "mod_alias" )


Add cgi-bin alias for doamin theos.in
alias.url = ( "/cgi-bin/" => "/home/lighttpd/theos.in/cgi-bin/" )

Browse all documents installed at /usr/share/doc/ directory with following alias:
alias.url = ( "/docs/" => "/usr/share/doc/" )
alias.url += ( "/stats/" => "/home/theos.in/http/webalizer/" )

Open a browser and type url http://theos.in/docs/ or http://your-domain.com/docs/

Shell script to watch the disk space

df displays the amount of disk space available on the file system containing each file name argument. If no file name is given, the space available on all currently mounted file systems is shown. Read man page of df if you are new to df command.


=> Find disk space using df

=> Filter out filesystem and find out the percentage of space using grep

=> Write a shell script

Step # 1: First get disk space:

$ df -H


Filesystem             Size   Used  Avail Use% Mounted on
/dev/hdb1               20G    14G   5.5G  71% /
tmpfs                  394M   4.1k   394M   1% /dev/shm
/dev/hdb5               29G    27G   654M  98% /nas/www

Step # 2: Next filter out filesystem and find out the percentage of space

$ df -H | grep -vE '^Filesystem|tmpfs|cdrom' | awk '{ print $5 " " $1 }'


71% /dev/hdb1
98% /dev/hdb5

Step # 3: Write a shell script

Above command displays field 5 and 1 of df command. Now all you need to do is write a script to see if the percentage of space is >= 90% (download script):

df -H | grep -vE '^Filesystem|tmpfs|cdrom' | awk '{ print $5 " " $1 }' | while read output;
  echo $output
  usep=$(echo $output | awk '{ print $1}' | cut -d'%' -f1  )
  partition=$(echo $output | awk '{ print $2 }' )
  if [ $usep -ge 90 ]; then
    echo "Running out of space \"$partition ($usep%)\" on $(hostname) as on $(date)" |
     mail -s "Alert: Almost out of disk space $usep%" you@somewhere.com

Setup Cron job

Save and install script as cronjob. Copy script to /etc/cron.daily/ (script downolad link)
# cp diskAlert /etc/cron.daily/
# chmod +x /etc/cron.daily/diskAlert

OR install as cronjob:
crontab -e

Write cronjob as per your requirement
10 0 * * * /path/to/diskAlert

Updated script version

Tony contributed and updated my script - You can exclude selected filesystem in case you don't want monitor all filesystems.

# set -x
# Shell script to monitor or watch the disk space
# It will send an email to $ADMIN, if the (free available) percentage of space is >= 90%.
# -------------------------------------------------------------------------
# Set admin email so that you can get email.
# set alert level 90% is default
# Exclude list of unwanted monitoring, if several partions then use "|" to separate the partitions.
# An example: EXCLUDE_LIST="/dev/hdd1|/dev/hdc5"
function main_prog() {
while read output;
#echo $output
  usep=$(echo $output | awk '{ print $1}' | cut -d'%' -f1)
  partition=$(echo $output | awk '{print $2}')
  if [ $usep -ge $ALERT ] ; then
     echo "Running out of space \"$partition ($usep%)\" on server $(hostname), $(date)" | \
     mail -s "Alert: Almost out of disk space $usep%" $ADMIN
if [ "$EXCLUDE_LIST" != "" ] ; then
  df -H | grep -vE "^Filesystem|tmpfs|cdrom|${EXCLUDE_LIST}" | awk '{print $5 " " $6}' | main_prog
  df -H | grep -vE "^Filesystem|tmpfs|cdrom" | awk '{print $5 " " $6}' | main_prog

Repairing ReiserFS file system with reiserfsck

Repairing ReiserFS file system with reiserfsck

The idea and commands in this article submitted by Jacques Wagener via email. In his own words, "After nuking my partition by accident (and through my stupidity) I was really disappointed in myself, especially in losing my bookmarks and rss-feeds". The following article is based upon our email communication. I am just putting them as an article.

We have already written about ext2/ext3 file repair using fsck and other utilities. Linux comes with different filesystems and different repair utilities. To repair a ReiserFS filesystem you need to run reiserfsck command, which is a checking tool for the ReiserFS filesystem (just like fsck command for ext2/ext3 file system).

Reiserfsck searches for a Reiserfs filesystem on a device, replays any necessary transactions, and either checks or repairs the file system. ReiserFS saves data or log in a special file for pending disk updates and later on it commit updates to disk resulting into very good filesystem consistency.

Step # 1: Install reiserfsck

You need to install reiserfsprogs package, which includes reiserfsck user level tools for ReiserFS filesystems.
# apt-get install reiserfsprogs

Or if you are using RedHat/Fedor Core Linux:
# yum install reiserfsprogs

Step # 2: Backup partition/disk

Take system down to runlevel 1.
# init 1

Unmount /dev/sda1 (if it is still mounted as read/write only):
# umount /dev/sda1; umount /dev/sda2

Before using any one of the following command you are strongly advised to make a backup copy of the whole partition using dd or ddrescue (recommended) command.
# ddrescue /dev/sda /dev/sdb

OR use dd command:
# dd if=/dev/sda of=/dev/sdb conv=noerror

Step # 3: Check filesystem consistency

Above command makes a backup of your drive. Next check filesystem consistency which will reports problem to you with the following command (assuming that /dev/sda1 is your partition):
# reiserfsck --check /dev/sda1

If you get an error Bad root block 0If you get following message:
Running with --rebuild-tree is required

Then you need to run following command rebuilds the entire filesystem tree using leaf nodes found on the device (this is nothing but indication of corrupted :
# reiserfsck --scan-whole-partition --rebuild-tree /dev/sda1


  • --scan-whole-partition: This option causes --rebuild-tree to scan the whole partition but not only the used space on the partition. You should always use this option with --rebuild-tree.
  • --check: Checks filesystem consistency and reports, but does not repair any corruption that it finds.
  • --rebuild-tree: This option rebuilds the entire filesystem tree using leaf nodes found on the device. Once you passed this option, do not stop or interrupt building operating.

Recovering corrupted superblock

Each file system has a superblock, which contains information about file system such as:

  • File system type
  • Size
  • Status
  • Information about other metadata structures

If this information lost, you are in trouble (data loss) so Linux maintains multiple redundant copies of the superblock in every file system.

During check (reiserfsck --check /dev/sda1) if you get an error superblock was missing, use following command to fix superblock:
# reiserfsck --rebuild-sb /dev/sda1


  • --rebuild-sb: This option recovers the superblock on a Reiserfs partition. Normally you only need this option if mount reports "read_super_block: can't find a reiserfs file system".

Caution: Do not run above command twice on same drive. You will damage your partition (data).

Final note

Next logical step is mount your partition /dev/sda1 and check for your data:
# mkdir -p /mnt/data
# mount /dev/sda1 /mnt/data
# cd /mnt/data
# ls
# ls lost+found/ -l

lost+found is a special directory where recovered files are kept by Linux/reiserfsck. You can examine these files and restore the data.

Better backup entire partition using tar or ssh session:
# tar cvf /dev/nst0 /mnt/data

OR use scp to dump data to remote system:
# scp -r /mnt/data you@other.server.com:/backup

See also:

If you are using hot swappable hard disk and created new partition using fdisk then you need to reboot Linux based system to get partition recognized. Without reboot you will NOT able to create filesystem on your newly created or modified partitions with the mke2fs command.

However with partprobe command you should able to create a new file system without rebooting the box. It is a program that informs the operating system kernel of partition table changes, by requesting that the operating system re-read the partition table.
[click to continue…]