BASH Shell: For Loop File Names With Spaces

Posted on in Categories File system, Linux, UNIX last updated November 16, 2013

BASH for loop works nicely under UNIX / Linux / Windows and OS X while working on set of files. However, if you try to process a for loop on file name with spaces in them you are going to have some problem. For loop uses $IFS variable to determine what the field separators are. By default $IFS is set to the space character. There are multiple solutions to this problem.

Understanding the Linux file system directories / hierarchy

Posted on in Categories Linux, RedHat/Fedora Linux, Sys admin, Tips, UNIX last updated August 24, 2007

I’ve already written about conceptual information regarding file system, especially data structure and related terms that help you become a successful system administrator. However I do get few emails asking about /opt directory or /usr or /lost+found directories and their purpose on the system.

Exploring Linux File System Hierarchy

A typical Linux system has the following directories:

=> / : This is the root directory.

=> /bin : This directory contains executable programs which are needed in single user mode and to bring the system up or repair it.

=> /boot : Contains static files for the boot loader. This directory only holds the files which are needed during the boot process.

=> /dev : Special or device files, which refer to physical devices such as hard disk, keyboard, monitor, mouse and modem etc

=> /etc : Contains configuration files which are local to the machine. Some larger software packages, like Apache, can have their own subdirectories below /etc i.e. /etc/httpd. Some important subdirectories in /etc:

  • /etc/skel : When a new user account is created, files from this directory are usually copied into the user’s home directory.
  • /etc/X11 : Configuration files for the X11 window system .
  • /etc/sysconfig : Important configuration file used by SysV script stored in /etc/init.d and /etc.rcX directories
  • /etc/cron.* : cron daemon configuration files which is used to execute scheduled commands

=> /home : Your sweet home to store data and other files. However in large installation yhe structure of /home directory depends on local administration decisions.

=> /lib : This directory should hold those shared libraries that are necessary to boot the system and to run the commands in the root filesystem.

=> /lib64 : 64 bit shared libraries that are necessary to boot the system and to run the commands in the root filesystem.

=> /mnt : This directory contains mount points for temporarily mounted filesystems

=> /opt : This directory should contain add-on packages such as install download firefox or static files

=> /proc : This is a mount point for the proc filesystem, which provides information about running processes and the kernel.

=> /root : This directory is usually the home directory for the root user.

=> /sbin : Like /bin, this directory holds commands needed to boot the system, but which are usually not executed by normal users, root / admin user specific commands goes here.

=> /tmp : This directory contains temporary files which may be deleted with no notice, such as by a regular job or at system boot up.

=> /usr : This directory is usually mounted from a separate partition. It should hold only sharable, read-only data, so that it can be mounted by various machines run ning Linux (useful for diskless client or multiuser Linux network such as university network). Programs, libraries, documentation etc. for all user-related programs.

=> /var : This directory contains files which may change in size, such as spool and log files.

=> /lost+found : Every partition has a lost+found in its upper directory. Files that were saved during failures are here, for e.g ext2/ext3 fsck recovery.

How do I get information about each and every directory from command prompt?

Above list just summaries important directories, you can get list by entering the following command to display description of the file system hierarchy (works under all other UNIX like oses such as HP-UX ):
$ man hier

List contents of directories in a tree-like format

ls command is basic tool for exploring file system. You can use tree command for a recursive directory listing. It produces a depth indented listing of files.
$ tree
Output:

|-- DIR_COLORS
|-- DIR_COLORS.xterm
|-- Muttrc
|-- Muttrc.local
|-- NetworkManager
|-- X11
|   |-- Xmodmap
|   |-- Xresources
|   |-- applnk
|   |-- fs
|   |   `-- config
.....
.......
..

find is another useful command to search for files in a directory hierarchy.

Further readings:

Advanced techniques for using the UNIX/Linux find command

Posted on in Categories Howto, Linux, News last updated February 18, 2008

IBM developerWorks published new article for Intermediate Linux/UNIX users. Explore the vast terrain of the UNIX file system with the find command. One of the most powerful and useful commands in the UNIX programmer’s repertoire is find. All flavors of UNIX have file systems that can contain thousands of files of many different types. With so many choices, locating a specific file, or set of files, can be difficult. The find command makes this task easier in many ways.

Read the article online.

Uninstall files installed from a source code tar ball on Linux or Unix

Posted on in Categories Debian Linux, File system, FreeBSD, GNU/Open source, Howto, Linux, RedHat/Fedora Linux, Solaris, Suse Linux, Tips, Ubuntu Linux, UNIX last updated October 26, 2015

Installing software from a source code is common practice in Linux and Unix world. Some time this is preferred method because it gives all power and flexibility you need to optimize your software such as MySQL, PHP, Apache and more. However, uninstalling files installed from a source code tar ball can be a big headache.

Two methods can be used to uninstall files:
Continue reading “Uninstall files installed from a source code tar ball on Linux or Unix”

Linux / UNIX: Finding and locating files with find command part # 2

Posted on in Categories CentOS, File system, FreeBSD, GNU/Open source, Howto, Linux, Linux desktop, OpenBSD, RedHat/Fedora Linux last updated August 24, 2007

In the first part we talked about find command basic usage.

Now let us see how to use find command
(a) To gain lots of useful information about users and their files

(b) Monitor and enhance the security of system using find command

Finding all set user id files

setuid (“suid”) and setgid are access right flags that can be assigned to files and directories on a Unix based operating system. They are mostly used to allow users on a computer system to execute binary executables with temporarily elevated privileges in order to perform a specific task.
# find / -perm +u=s
OR
# find / -perm +4000

See also, shell script to find all programs and scripts with setuid set on.

Finding all set group id files

# find / -perm +g=s
OR
# find / -perm +2000

See also, shell script to find all programs and scripts with setgid bit set on.

Finding all large directories

To find all directories taking 50k (kilobytes) blocks of space. This is useful to find out which directories on system taking lot of space.
# find / -type d -size +50k
Output:

/var/lib/dpkg/info
/var/log/ksymoops
/usr/share/doc/HOWTO/en-html
/usr/share/man/man3

Finding all large files on a Linux / UNIX

# find / -type f -size +20000k
Output:

var/log/kern.log
/sys/devices/pci0000:00/0000:00:02.0/resource0
/sys/devices/pci0000:00/0000:00:00.0/resource0
/opt/03Jun05/firefox-1.0.4-source.tar.bz2

However my favorite hack to above command is as follows:
# find / -type f -size +20000k -exec ls -lh {} \; | awk '{ print $8 ": " $5 }'

/var/log/kern.log: 22M
/sys/devices/pci0000:00/0000:00:02.0/resource0: 128M
/sys/devices/pci0000:00/0000:00:00.0/resource0: 256M
/opt/03Jun05/firefox-1.0.4-source.tar.bz2: 32M

Above command will find all files block size greater than 20000k and print filename followed by the file size. Output is more informative as compare to normal find command output :D

Linux: Finding and Locating files with find command part # 1

Posted on in Categories CentOS, File system, Linux, Linux desktop, RedHat/Fedora Linux last updated August 24, 2007

Many newcomers find it difficult use the find command at shell prompt under Linux / *BSD or Solairs UNIX oses. Find is nifty tool on remote server where UNIX admin can find out lot of information too. Desktop users may find handy GNOME Search tool as a utility for finding files on system. Find command can perform a search based on a variety of search constraints. It searches through one or more directory tree(s) of a filesystem, locating files based on some user-specified criteria. By default, find returns all files below the current working directory. Further, find allows the user to specify an action to be taken on each matched file. Thus, it is an extremely powerful program for applying actions to many files. It also supports regexp matching.

GNOME Search Tool GUI Program

GNOME Search Tool is a utility for finding files on your system. To perform a basic search, you can type a filename or a partial filename, with or without wildcards. You can start this program from menus or by typing following command at shell prompt:
$ gnome-search-tool &
Internally GNOME Search Tool uses the find, grep, and locate UNIX commands. The case sensitivity of the search depends on your operating system. For example, on Linux, the find, grep, and locate commands support the -i option, so all searches are case-insensitive.

the GNOME Linux / UNIX GUI Search Tool(click to enlarge)

Find command syntax

find {search-path} {file-names-to-search} {action-to-take}

Where,

  • search-path : Define search path (default current directory). For example search in /home directory.
  • file-names-to-search : Name of the file you wish to find. For example all c files (*.c)
  • action-to-take : Action can be print file name, delete files etc. Default action is print file names.

Find command examples

Let us try out some examples.

Finding files and printing their full name

You wish to find out all *.c (all c source code) files located under /home directory, enter:
$ find /home -name "*.c"

You would like to find httpd.conf file location:
$ find / -name httpd.conf

Finding all files owned by a user

Find out all files owned by user vivek:
# find / -user vivek

Find out all *.sh owned by user vivek:
# find / -user vivek -name "*.sh"

Finding files according to date and time

Files not accessed in a time period – It is useful to find out files that have or have not been accessed within a specified number of days. Following command prints all files not accessed in the last 7 days:
# find /home -atime +7

  • -atime +7: All files that were last accessed more than 7 days ago
  • -atime 7: All files that were last accessed exactly 7 days ago
  • -atime -7: All files that were last accessed less than7 days ago

Finding files modified within a specified time – Display list of all files in /home directory that were not last modified less than then days ago.
# find /home -mtime -7

Finding newer (more recently) modified files

Use -newer option to find out if file was modified more recently than given file.
# find /etc/apache-perl -newer /etc/apache-perl/httpd.conf

Finding the most recent version of file

It is common practice before modifying the file is copied to somewhere in system. For example whenever I modify web server httpd.conf file I first make backup. Now I don’t remember whether I had modified the /backup.conf/httpd.conf or /etc/apache-perl/httpd.conf. You can use the find command as follows (tip you can also use ls -l command):
find / -name httpd.conf -newer /etc/apache-perl/httpd.conf

Locate command

The locate command is often the simplest and quickest way to find the locations of files and directories on Linux and other Unix-like operating systems.

For example, the following command uses the star wildcard to display all files on the system that have the .c filename extension:
# locate "*.c"

Further readings

  • Read find and locate command man page for more information.