BASH Shell: For Loop File Names With Spaces

by Vivek Gite · 31 comments

BASH for loop works nicely under UNIX / Linux / Windows and OS X while working on set of files. However, if you try to process a for loop on file name with spaces in them you are going to have some problem. for loop uses $IFS variable to determine what the field separators are. By default $IFS is set to the space character. There are multiple solutions to this problem.

Set $IFS variable

Try it as follows:

#!/bin/bash
SAVEIFS=$IFS
IFS=$(echo -en "\n\b")
for f in *
do
  echo "$f"
done
IFS=$SAVEIFS

#!/bin/bash
SAVEIFS=$IFS
IFS=$(echo -en "\n\b")
# set me
FILES=/data/*
for f in $FILES
do
  echo "$f"
done
# restore $IFS
IFS=$SAVEIFS

More examples using $IFS and while loop

Now you know that if the field delimiters are not whitespace, you can set IFS. For example, while loop can be used to get all fields from /etc/passwd file:

....
while IFS=: read userName passWord userID groupID geCos homeDir userShell
do
      echo "$userName -> $homeDir"
done < /etc/passwd

Using old good find command to process file names

To process the output of find with a command, try as follows:

find . -print0 | while read -d $'\0' file
do
  echo -v "$file"
done

Try to copy files to /tmp with spaces in a filename using find command and shell pipes:

find . -print0 | while read -d $'\0' file; do cp -v "$file" /tmp; done

Processing filenames using an array

Sometimes you need read a file into an array as one array element per line. Following script will read file names into an array and you can process each file using for loop. This is useful for complex tasks:

#!/bin/bash
DIR="$1"
 
# failsafe - fall back to current directory
[ "$DIR" == "" ] && DIR="." 
 
# save and change IFS
OLDIFS=$IFS
IFS=$'\n' 
 
# read all file name into an array
fileArray=($(find $DIR -type f))
 
# restore it
IFS=$OLDIFS
 
# get length of an array
tLen=${#fileArray[@]}
 
# use for loop read all filenames
for (( i=0; i<${tLen}; i++ ));
do
  echo "${fileArray[$i]}"
done

Playing mp3s with spaces in file names

Place following code in your ~/.bashrc file:

mp3(){
	local o=$IFS
	IFS=$(echo -en "\n\b")
	/usr/bin/beep-media-player "$(cat  $@)" &
	IFS=o
}

Keep list of all mp3s in a text file such as follows (~/eng.mp3.txt):

/nas/english/Adriano Celentano - Susanna.mp3
/nas/english/Nick Cave & Kylie Minogue - Where The Wild Roses Grow.mp3
/nas/english/Roberta Flack - Kiling Me Softly With This Song.mp3
/nas/english/The Beatles - Girl.mp3
/nas/english/John Lennon - Stand By Me.mp3
/nas/english/The Seatbelts, Cowboy Bebop - 01-Tank.mp3

To play just type:
$ mp3 eng.mp3.txt

Featured Articles:

Want to read Linux tips and tricks, but don't have time to check our blog everyday? Subscribe to our daily email newsletter to make sure you don't miss a single tip/tricks. Subscribe to our weekly newsletter here!

{ 31 comments… read them below or add one }

1 Sergio 12.09.08 at 9:44 pm

Hi, I don’t fully understand how this…

BASH for loop works nicely under UNIX / Linux / Windows and OS X

works nicely under Windows…

Oh! btw, thank you.

2 Vivek Gite 12.09.08 at 10:00 pm

Install cygwin from http://www.cygwin.com/

Install bash and your are done!

3 Jeff Schroeder 12.10.08 at 4:02 am

For the shell equivalent of python’s splitlines, I always did it like this:

oldifs="$IFS"
IFS="
"
for line in $(< line-with-spaces-between-fields.txt); do
   ...
done
IFS="$oldifs"

or while read ... instead of for when the list exceeds $(getconf ARG_MAX)

Your way works too, but is more typing

4 Amr Hamdy 12.10.08 at 4:06 am

Thanks really so much,
It’s so useful and I think it’d save me a lot of time and help me do better shell coding..

5 Benjamin Schmidt 12.10.08 at 8:03 am

This is a great blog!

Instead of using following:

find . -print0 | while read -d $'' file
do
  echo -v "$file"
done

I just use:

find . | while read file
do
  echo "$file"
done

Never had problems with it. A filename can impossible contain newline-chars, or am I wrong?

Best regards,
Benjamin Schmidt

6 Vivek Gite 12.10.08 at 8:59 am

@Jeff/Benjamin

Thanks for sharing your code and ideas.

7 chika.tambun 12.10.08 at 12:17 pm

i’d love to play music with mplayer… through cli
let say that i have song/music folder with many directory inside, how i can play with mplayer with single command. Actually i have one bash script, but i got trouble when facing white space… mplayer wouldn’t play the file

[songRequest.sh]
for i in ` find -name “*mp3″`
do
mplayer $i
done

any help will be appreciated

8 Vivek Gite 12.10.08 at 12:21 pm

Try

find .  -iname "*.mp3" -print0 | while read -d $'\0' file
do
  mplayer "$file"
done

9 Chris F.A. Johnson 12.10.08 at 3:40 pm

> A filename can impossible contain newline-chars, or am I wrong?

A filename can contain newlines. It can contain any character except a slash (/) or an ASCII NUL (0×0).

10 Chris F.A. Johnson 12.10.08 at 3:45 pm

# use for loop read all filenames
> for (( i=0; i do
> echo “${fileArray[$i]}”
> done

If all you want to do is print the contents of the array:

printf "%s\n" "${fileArray[@]}"

To loop through them, use:

for i in "${fileArray[@]}" do : ... whatever done

11 chika.tambun 12.11.08 at 1:24 pm

great… vivek.gite ^^

your script just work on one file… then mplayer killed immediately.
yeah just playing one file only(there are many left).

on playing progress i can’t get control of mplayer, even i can’t turn up/down volume, go to the next song, etc.

all i can do is Ctrl+C

btw thanks

12 Vivek Gite 12.11.08 at 2:12 pm

I guess you can try
mplayer *.mp3

Or send all files to .playlist.txt
ls -1 > .playlist.txt mplayer -playlist .playlist.txt

13 mhernandez 12.11.08 at 3:30 pm

All this information is very useful, but I think that needing it is sympton: ¿shouldn’t you use another language instead?

Nice post, though

14 Chris F.A. Johnson 12.11.08 at 9:08 pm

“All this information is very useful, but I think that needing it is sympton: ¿shouldn’t you use another language instead?”

Why? It’s straightforward shell scripting.

15 mhernandez 12.11.08 at 9:46 pm

opendir(DIR, “.”) or die(“Error opening directory”);
my @readFiles = readdir(DIR);
foreach (@readFiles) {
…

16 Kyle Brandt 12.16.08 at 7:27 pm

I don’t think there is a need to reset the IFS variable, all you really need to do to avoid problems with word splitting is to put the variables that appear in the loop body in double quotes (I find this to be a good default habit). Once in double quotes the shell will consider whatever the variable expands to as a single argument and pass it to the program as such.

For example:
for i in *.mp3; do rm “$i”; done

mhernandez: Its not a problem with the shell, its just how it works, the bash shell is based on word splitting.

17 Dean Faulkner 01.07.09 at 4:57 pm

You can reset the IFS with
unset IFS
this way, you do not need to save the state and then recall it from a variable.

18 Jim Eberle 01.10.09 at 3:47 am

Kyle is correct. No need to mess w/ your IFS var.
Your 2nd example is problematic in a very subtle way

FILES=/data/*

does not set the variable ‘FILES’ to the names of all the files in /data, it sets it to ‘/data/*’. You can see this w/:

declare -p FILES

A very useful technique in all this, is to employ the ’set’ builtin:

cd ..; set *.txt; cd ~- for f; do echo "../$f" "$f" done

This lets you scoop up a file list (w/ nasty names and all), and then move somewhere else to process them.

19 joel 02.10.09 at 5:30 am

i am doing a project in unix shell scripting.
so my project have messages to be displayed. my problem is that how to add a beep sound to the messages …can any help me to include that also with the coding??pls..its very urgent…i dont know the shell codings…its for my project

20 Reid 02.19.09 at 5:17 pm

Chris F.A. Johnson has it right on the nose.

use a quoted @-array: <code“${fileArray[@]}”

A quoted array with an @ subscript expands to a list of quoted elements of that array.

No IFS manipulation required!

Reid

21 DoXuS 07.21.09 at 11:51 am

Thanks man, my code is now only 40 lines long, not 136 like before xD^^
nice blog

22 fvw 08.26.09 at 4:48 am

mp3(){
local o=$IFS
IFS=$(echo -en “\n\b”)
/usr/bin/beep-media-player “$(cat $@)” &
IFS=o
}
“$(cat $@)” if you use this I think the IFS down work …
I think you want $(cat “$@”)

23 Rich 09.02.09 at 8:38 pm

Thank you! I found this post most useful.

24 gimi 10.01.09 at 9:06 pm

Instead of telling echo to print a char for new line and (probably) bell and then *not* to print a new line, one can just use echo’s default new line output :)

IFS=`echo`

Thanks for your share!

25 cfajohnson 10.07.09 at 9:45 am

Why “IFS=`echo`”? Command substitution is slow (and unnecessary).

IFS=’
‘

IFS=$’\n’ ## bash

26 Brian 11.13.09 at 5:47 am

Amazing! Your notation for creating an array from:
> fileArray=($(find $DIR -type f))
is stunning! Thanks for the excellent writeup.

27 xiaoyuhum 11.26.09 at 7:07 am

I just use as following
with double quotes
[code]
for m in "`ls`"; do echo "$m" ; done
[/code]

28 cfajohnson 12.02.09 at 10:36 pm

for m in "`ls`"; do echo "$m" ; done

Why?

The for command will only see one argument.

Try this:

n=1 for m in "`ls`" do echo "$n" echo "$m" n=$(( $n + 1 )) done

You’ll see that it only goes through the loop once.

You don’t need a loop for that, and you don’t need ls:

printf "%s\n" *

29 Shreyas Kulkarni 12.19.09 at 5:52 pm

excellent post. exactly what i was looking for. i had been facing this space-field-seperator problem for quite a sometime today. had tried many options to circumvent it, but in vain. will try this $IFS approach now.

thanks a lot for sharing.

shreyas

30 Chris Thiessen 02.08.10 at 8:09 pm

My favorite solution, though it can theoretically overmatch:

find . | sed -e “s/ /?/g”
ls | sed -e “s/ /?/g”

Then you can do things like greps, for loops, whatever equally well:
grep hello $(ls | sed -e “s/ /?/g”)
for i in $(find . | sed -e “s/ /?/g) do; ls -l “$i”; done

31 cfajohnson 02.09.10 at 8:07 am

“grep hello $(ls | sed -e “s/ /?/g”)”

You don’t need sed or ls; use filename expansion:

grep hello *

“for i in $(find . | sed -e “s/ /?/g) do; ls -l “$i”; done”

That will fail if any filenames contain whitespace.

It is also extremely inefficient to call ls in a loop.