Linux Optimize Directories ( File Access Time ) in ext3 Filesystem

by on January 5, 2008 · 19 comments· LAST UPDATED January 9, 2008

in , ,

Q. I'm using CentOS Linux 5 server with ext3 on /disk2 (/dev/sdb1). How do I improve my file server performance. I've many large files in directories and how do I speed up access time?

A. Use e2fsck command. The -D option causes e2fsck to try to optimize all directories, either by reindexing them if the filesystem supports directory indexing, or by sorting and compressing directories for smaller directories, or for filesystems using traditional linear directories. It improves performance by using hashed binary trees.

ext3 File Tuning / Directory Optimization

Login as the root user

Take down system to single user mode and unmount file system:
# umount /dev/sdb1

Type the following command:

WARNING! Running e2fsck on a mounted filesystem may cause SEVERE filesystem damage. Always unmount file system.

Now run e2fsck on unmounted file system:
# e2fsck -D -f /dev/sdb1
Output:

e2fsck 1.39 (29-May-2006)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 3A: Optimizing directories
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/disk2: ***** FILE SYSTEM WAS MODIFIED *****
/disk2: 39/26104 files (17.9% non-contiguous), 20045/104388 blocks

Mount file system:
# mount /dev/sdb1

Now your should see improvements while accessing files. For more information please read e2fsck man page:
man e2fsck

TwitterFacebookGoogle+PDF versionFound an error/typo on this page? Help us!

{ 19 comments… read them below or add one }

1 momo January 5, 2008 at 6:08 pm

I get: ‘umount: /: device is busy’ (logged on through ssh).
What can I do?

Reply

2 nixCraft January 5, 2008 at 7:08 pm

momo,

You need to take system to a single user mode and this cannot be done using ssh. You need either physical access to computer or use KVM remote access.

Reply

3 Nilesh January 6, 2008 at 2:04 am

for this, either we need more than 1 partition or the rescue cd of the OS!

because, if we have one partition, we can’t umount it for e2fsck. using rescue cd is the only option available to single partition systems (like me).

sometimes even the rescue cd method fails. :D

Reply

4 Donald January 6, 2008 at 5:59 am

Is there a reason that you umount and fsck sdb1 and mount sdb2 or is that a typo?

Reply

5 nixCraft January 6, 2008 at 8:27 am

Running e2fsck on a mounted filesystem corrupts file system for sure. So you need to unmount it.

Reply

6 Donald January 6, 2008 at 3:54 pm

No, my point was that you umount/fsck a different filesystem than you mount at the end (sdb1, sdb2).

Reply

7 nixCraft January 9, 2008 at 10:16 am

Now I feel stupid :(

Thanks for the heads up. The faq has been updated.

Reply

8 dennisraj January 9, 2008 at 5:26 pm

I am running redhat linux 8. i upgraded glibc 2.2 to glibc 2.3. i upgarded all the dependencies also.i took the glibc 2.3 from redhat linux 9. Now the problem is that i cant able to install or uninstall any rpm. It is showing segmentation fault. Please tell me what i have to do next.?? Waiting for your reply?

Reply

9 nixCraft January 9, 2008 at 6:32 pm

Redhat 9?, No one can fix your problem. Get latest version http://centos.org/ or http://fedoraproject.org/

Reply

10 Nilesh January 12, 2008 at 5:46 pm

dennisraj,
solution is to run fsck -a on the filesystem.

it will ask you to unmount. you could do it if you have two devices.

you could run it on a mounted filesystem also but then don’t blame me if something goes wrong.

nothing should happen on running it on a mounted filesystem as i have done that many times.

But REMEMBER, DO NOT INTERRUPT THE PROCESS IN BETWEEN. IF YOU DO SO, LINUX IS CRASHED (RECOVERY=REINSTALL).

Reply

11 Whyworry February 1, 2008 at 2:24 pm

Get a life dennisraj. Stop asking questions period!!

If you want to stay on RH8, then sort your own problem out and don’t trouble other people. Also I can’t see your claim of RH8 to be credible, I’m quite sure your system is unique to yourself.

Additionally stop hijacking articles by putting unrelated comments/questions.

Sort it out and move on.

Reply

12 Mauriat February 1, 2008 at 2:55 pm

“Now your should see improvements while accessing files.”

I am really curious. What improvements should be seen? Is there some measure you can use?

And since this is “reindexing”, would the process need to be run on some regular basis?

Reply

13 Alex Chekholko February 1, 2008 at 4:05 pm

While running fsck on your system is not a bad idea, it is not clear to me why it would improve performance. The author merely states that it does, without backing that statement up with any facts. Searching google for ‘”e2fsck -D” performance’ brings up a couple of other interesting articles, but no hard numbers.

More importantly, directory indexing is turned on by default already on modern distros, just take a look at /etc/mke2fs.conf (on my CentOS5 system), so the -D option really doesn’t do anything.

Reply

14 Raj February 1, 2008 at 4:23 pm

Mauriat,
And since this is “reindexing”, would the process need to be run on some regular basis?
Do it only once and it will turn it on, edit /etc/mke2fs.conf to set it on.

Now your should see improvements while accessing files.
Run before:
dd if=/dev/zero of=/path/2/file bs=16384 count=131072
time cat /path/2/file > /dev/null

Reboot (so cache get away); turn on indexing and rerun above commands to measure performance.

Alex,
Directory Indexing increases the speed of reads and writes as they are pre-indexed – so the read speed should increase. This is based upon own experience. Yes, some distro does turn it on by default so no need to execute e2fsck.

http://wiki.centos.org/HowTos/Disk_Optimization
Kernel ext3 doc has more information

Reply

15 Raj February 1, 2008 at 4:32 pm

Just a small addition,

Run dd using for loop and read output to get real stats out of this.

Hope this helps someone :)

Reply

16 gus3 February 1, 2008 at 6:13 pm

It’s possible to fsck the root filesystem, if it is mounted read-only. After “telinit S”, and with all other filesystems unmounted, type:

# mount -o remount,ro -t ext3 /dev/[device] /

This will leave it accessible, but no files may be changed, and no new files may be created.

Then, run the fsck -D command, and reboot immediately upon completion.

Reply

17 Bryan February 4, 2008 at 10:31 pm

Interesting tip. Since fsck runs periodically after a certain number of intervals (specified by tune2fs -i) I was wondering if there was a way to run the -D flag at that time?

Reply

18 AMITKUMAR May 15, 2008 at 6:24 am

how to design ext3 linux file system form the scratch

Reply

19 Alan Aversa September 9, 2012 at 5:33 am

Do console login and run “init 1″ before doing any defragging.

Reply

Leave a Comment

Tagged as: , , , , , , , , , , , , , , , , , , , ,

Previous Faq:

Next Faq: