≡ Menu

filesystem

When you work in tech support department and deal with inexperienced clients debugging problems turns into a nightmare. As a sysadmin, you won't become too paranoid if less experienced people have root-access. As a consultant, you won’t feel isolated if you don’t have remote access to your systems. As a support engineer, you won’t become frustrated if a customer has fiddled around with some important config file and you have to find which. As a performance tuner, you can capture the state of the system configuration in between performance tests/benchmarks.

Luckily, some nifty tools can create a system's hardware and software configuration snapshot. This kind of information is valuable asset while troubleshooting problems.

dconf (System config collector) is one of such tool. It allows to take your system configuration with you on the road, compare identical systems (like nodes in a cluster) to troubleshoot HW or SW problems, indeed a lifesaver.

Dconf is also useful in projects where you have to manage changes as a team. Dconf can run periodically and send out system changes to a list of email addresses so that they can be revised and discussed in group.

You can customize your dconf configuration for specific needs, like making a profile of your web server’s hardware or copy specific software configuration files to send out or compare with other systems.

As a sysadmin, you will not become too paranoid if less experienced people have root-access. As a consultant, you will not feel isolated if you do not have remote access to your systems. As a support engineer, you will not become frustrated if a customer has fiddled around with some important config file and you have to find which. As a performance tuner, you can capture the state of the system configuration in between performance tests/benchmarks.

Install dconf

If you are using Debian / Ubuntu Linux then type the command:
# apt-get install dconf
You can download Dconf for RedHat or Suse Linux here

Create a system's hardware and software configuration snapshot

Once installed you can simply create a snapshot using dconf command:
# dconf
It will write snapshot in /var/log/dconf/ directory. To view current snapshot info, enter:
# zcat /var/log/dconf/dconf-$HOSTNAME-latest.log.gz
To check the latest changes against the previous snapshot:
# zdiff -u /var/log/dconf/dconf-$HOSTNAME-previous.log.gz /var/log/dconf/dconf-$HOSTNAME-latest.log.gz

See also:

Why command df and du reports different output?

You will never notice something like this on FreeBSD or Linux Desktop home system or your personal UNIX or Linux workstation. However, sometime on a production UNIX server you will notice that both df (display free disk space) and du (display disk usage statistics) reporting different output. Usually df will output a bigger disk usage than du.

If Linux or UNIX inode is deallocated you will see this problem. If you are using clustered system (file system such as GFS) you may see this scenario commonly.

Note following examples are FreeBSD and GNU/Linux specific.

Following is normal output of df and du for /tmp filesystem:
# df -h /tmp
Output:

Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/ad0s1e    496M     22M    434M     5%    /tmp

Now type du command:
# du -d 0 -h /tmp/
Output:

22M    /tmp/

Why is there a mismatch between df and du outputs?

However, some time it reports different output (a bigger disk usage), for example:
# df -h /tmp/
Output:

Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/ad0s1e    496M     39M    417M     9%    /tmp

Now type du command:
# du -d 0 -h /tmp/
Output:

 22M    /tmp/

As you see, both df and du reporting different output. Many new UNIX admin get confused with output (39M vs 22M).

Open file descriptor is main causes of such wrong information. For example if file called /tmp/application.log is open by third party application OR by a user and same file is deleted, both df and du reports different output. You can use lsof command to verify this:
# lsof | grep tmp
Output:

bash   594  root  cwd   VDIR  0,86      512      2 /tmp
bash   634  root  cwd   VDIR  0,86      512      2 /tmp
pwebd  635  root  cwd   VDIR  0,86      512      2 /tmp
pwebd  635  root  3rW   VREG  0,86 17993324     68 /tmp (/dev/ad0s1e)
pwebd  635  root   5u   VREG  0,86        0     69 /tmp (/dev/ad0s1e)
lsof   693  root  cwd   VDIR  0,86      512      2 /tmp
grep   694  root  cwd   VDIR  0,86      512      2 /tmp

You can see 17993324K file is open on /tmp by pwebd (our in house software) but deleted accidentally by me. You can recreate above scenario in your Linux, FreeBSD or Unixish system as follows:

First, note down /home file system output:
# df -h /home
# du -d 0 -h /home

If you are using Linux then use du as follows:
# du -s -h /tmp

Now create a big file:
# cd /home/user
# cat /bin/* >> demo.txt
# cat /sbin/* >> demo.txt

Login on other console and open file demo.txt using vi text editor:
# vi /home/user/demo.txt

Do not exit from vi (keep it running).

Go back to another console and remove file demo.txt
# rm demo.txt
Now run both du and df to see the difference.
# df -h /home
# du -d 0 -h /home

If you are using Linux then use du as follows:
# du -s -h /tmp

Login to another terminal and close vi.

Now close the vi and the root cause of the problem should be resoled, the du and df outputs should be correct.

In the first part we talked about find command basic usage.

Now let us see how to use find command
(a) To gain lots of useful information about users and their files

(b) Monitor and enhance the security of system using find command

Finding all set user id files

setuid ("suid") and setgid are access right flags that can be assigned to files and directories on a Unix based operating system. They are mostly used to allow users on a computer system to execute binary executables with temporarily elevated privileges in order to perform a specific task.
# find / -perm +u=s
OR
# find / -perm +4000

See also, shell script to find all programs and scripts with setuid set on.

Finding all set group id files

# find / -perm +g=s
OR
# find / -perm +2000

See also, shell script to find all programs and scripts with setgid bit set on.

Finding all large directories

To find all directories taking 50k (kilobytes) blocks of space. This is useful to find out which directories on system taking lot of space.
# find / -type d -size +50k
Output:

/var/lib/dpkg/info
/var/log/ksymoops
/usr/share/doc/HOWTO/en-html
/usr/share/man/man3

Finding all large files on a Linux / UNIX

# find / -type f -size +20000k
Output:

var/log/kern.log
/sys/devices/pci0000:00/0000:00:02.0/resource0
/sys/devices/pci0000:00/0000:00:00.0/resource0
/opt/03Jun05/firefox-1.0.4-source.tar.bz2

However my favorite hack to above command is as follows:
# find / -type f -size +20000k -exec ls -lh {} \; | awk '{ print $8 ": " $5 }'

/var/log/kern.log: 22M
/sys/devices/pci0000:00/0000:00:02.0/resource0: 128M
/sys/devices/pci0000:00/0000:00:00.0/resource0: 256M
/opt/03Jun05/firefox-1.0.4-source.tar.bz2: 32M

Above command will find all files block size greater than 20000k and print filename followed by the file size. Output is more informative as compare to normal find command output :D

Many newcomers find it difficult use the find command at shell prompt under Linux / *BSD or Solairs UNIX oses. Find is nifty tool on remote server where UNIX admin can find out lot of information too. Desktop users may find handy GNOME Search tool as a utility for finding files on system. Find command can perform a search based on a variety of search constraints. It searches through one or more directory tree(s) of a filesystem, locating files based on some user-specified criteria. By default, find returns all files below the current working directory. Further, find allows the user to specify an action to be taken on each matched file. Thus, it is an extremely powerful program for applying actions to many files. It also supports regexp matching.

GNOME Search Tool GUI Program

GNOME Search Tool is a utility for finding files on your system. To perform a basic search, you can type a filename or a partial filename, with or without wildcards. You can start this program from menus or by typing following command at shell prompt:
$ gnome-search-tool &
Internally GNOME Search Tool uses the find, grep, and locate UNIX commands. The case sensitivity of the search depends on your operating system. For example, on Linux, the find, grep, and locate commands support the -i option, so all searches are case-insensitive.

the GNOME Linux / UNIX GUI Search Tool(click to enlarge)

Find command syntax

find {search-path} {file-names-to-search} {action-to-take}

Where,

  • search-path : Define search path (default current directory). For example search in /home directory.
  • file-names-to-search : Name of the file you wish to find. For example all c files (*.c)
  • action-to-take : Action can be print file name, delete files etc. Default action is print file names.

Find command examples

Let us try out some examples.

Finding files and printing their full name

You wish to find out all *.c (all c source code) files located under /home directory, enter:
$ find /home -name "*.c"

You would like to find httpd.conf file location:
$ find / -name httpd.conf

Finding all files owned by a user

Find out all files owned by user vivek:
# find / -user vivek

Find out all *.sh owned by user vivek:
# find / -user vivek -name "*.sh"

Finding files according to date and time

Files not accessed in a time period – It is useful to find out files that have or have not been accessed within a specified number of days. Following command prints all files not accessed in the last 7 days:
# find /home -atime +7

  • -atime +7: All files that were last accessed more than 7 days ago
  • -atime 7: All files that were last accessed exactly 7 days ago
  • -atime -7: All files that were last accessed less than7 days ago

Finding files modified within a specified time – Display list of all files in /home directory that were not last modified less than then days ago.
# find /home -mtime -7

Finding newer (more recently) modified files

Use -newer option to find out if file was modified more recently than given file.
# find /etc/apache-perl -newer /etc/apache-perl/httpd.conf

Finding the most recent version of file

It is common practice before modifying the file is copied to somewhere in system. For example whenever I modify web server httpd.conf file I first make backup. Now I don't remember whether I had modified the /backup.conf/httpd.conf or /etc/apache-perl/httpd.conf. You can use the find command as follows (tip you can also use ls -l command):
find / -name httpd.conf -newer /etc/apache-perl/httpd.conf

Locate command

The locate command is often the simplest and quickest way to find the locations of files and directories on Linux and other Unix-like operating systems.

For example, the following command uses the star wildcard to display all files on the system that have the .c filename extension:
# locate "*.c"

Further readings

  • Read find and locate command man page for more information.