UNIX: Loop Through Files In A Directory

by on July 1, 2010 · 3 comments· LAST UPDATED July 1, 2010

in

How do I loop through files in a directory under UNIX like operating systems?

The most simplest form is as follows:

 
for file in /path/to/file1.txt /path/to/file2.txt /path/to/file3.txt
do
 # do something on $file
 cat "$file"
done
 

You can directly process all command line args:

 
for file in $*
do
 # do something on $file
 [ -f "$file" ] && cat "$file"
done
 

OR simply process all *.css file in the current directory:

 
for file in *.css
do
 # do something on "$file"
 cat "$file" >> /var/www/cdn.example.com/cache/large.css
done
 

You can also read file names stored in a text file called delete.txt (note read with -r and IFS which will take care of file with spaces):

 
while IFS= read -r f <&3;
do
      #do something with "$f"
      rm -f "$f"
done 3< delete.txt
 

Make sure you always put $f or $file in double quote. Here is another sample script it will go through /home/wwwdata/{example.com,example.net,nixcraft.com} and process all files using for loop:

#!/bin/bash
# sync all domains to backup server at midnight 
domains="example.com example.net nixcraft.com cyberciti.biz"
me="${0##*/}"
now=$(date +"%d-%m-%Y_%S")
log="/tmp/${me}.${now}"
latest="/tmp/latest"
logdata(){
	local f="$1"
	local d="$2"
	[[ "$d" != "" ]] &&      echo "                            $d"
	[[ "$f" == "start" ]] && echo "--------------------------------------------------------------"
	[[ "$f" == "end" ]] &&   echo "=============================================================="
 
}
source /usr/local/nixcraft/mgmt/ssh/.keychain/$HOSTNAME-sh
for d in $domains
do
	logdata "start" "$d @ $(date)"
        [ -d "/home/wwwdata/$d/" ] && { 	cd "/home/wwwdata/$d/";
 	/usr/bin/rsync  --exclude='cache/cache-*'\
			--exclude '.bash_history' \
			--exclude '.viminfo' \
			--exclude 'cache/*_mutex.lock' \
			--exclude 'broken-link-checker*' \
                        --exclude 'tmp/*'
			-a --delete . backup@nasbox.nixcraft.net.in:/raid6/$HOSTNAME/ ;
         }
	logdata "end" "$d @ $(date)"
done &> $log
[ -f $latest ] && /bin/rm -f $latest
ln -s $log $latest
mail -s "Backup $HOSTNAME" admin@clients.nixcraft.net.in < $latest
 

See also:

TwitterFacebookGoogle+PDF versionFound an error/typo on this page? Help us!

{ 3 comments… read them below or add one }

1 Benjamin July 2, 2010 at 1:14 am

For most cases the -exec option of find is very helpful:
find . -name ‘*.css’ -exec cat {} >> large.css \;

# delete certain files younger than 60 minutes:
find /wherever/you/want -type f -name ‘*substring*’ -mmin -60 -delete

Reply

2 nixCraft July 2, 2010 at 10:01 am

Yes, find command is really useful, especially -print0 find option and read command to process filenames with spaces:

find . -print0 | while read -d $'\0' file; do cp -v "$file" /tmp; done

Reply

3 yoander (sedlav) July 2, 2010 at 1:29 pm

Another way, using xargs
find . -print0 -type f | xargs -0 -I {} cp -v {} /tmp

Reply

Leave a Comment

Tagged as: , , , , , , , , , , , , , , , , ,

Previous Faq:

Next Faq: