How Do I Replace A Text String In Many Files At Once?

Posted on in Categories Linux, Shell scripting, Tips, UNIX last updated November 12, 2005

The replace command is a string-replacement utility. It changes strings in place in files or on the standard input. This command uses a finite state machine to match longer strings first. It can be used to swap strings. This command is similar to the Perl -pie syntax or sed (stream editor) command.

Please note that the replace command is part of is MySQL database system. If you don’t have MySQL installed, you don’t have replace command.

Syntax

replace OLD-STRING NEW-STRING < INPUT-FILE > OUTPUT-FILE

Examples

To replace all occurrences of word UNIX with Linux, enter:
$ replace UNIX Linux < oldfile > newfile

The replace command can be used in a pipeline, run:
$ cat /etc/passwd | replace : '|'

You can skip the cat command, enter:
$ replace : '|' < /etc/passwd

It also supports few special characters in string replacement:

  • \^ : Match start of line.
  • $ : Match end of line.

How Do I Update All *.txt Files At Once?

You use bash for loop as follows:

#!/bin/bash
for f in /path/to/*.txt
do
   replace UNIX Linux < "$f" > "$f.new"
done

The replace command does not understand regular expression. To use regular expression try the sed command or Perl.

Sed Command Example

To replace all occurrences of word UNIX with Linux using the sed command, enter:

sed 's/UNIX/Linux/g' < input.file > output.file

OR

sed -i 's/UNIX/Linux/g' input.file

OR use bash shell for loop as follows to update all *.doc files at once:

#!/bin/bash
for f in /path/to/*.doc
do
   sed -i  's/UNIX/Linux/g' "$f"
done

Updated for accuracy!

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin and a trainer for the Linux operating system/Unix shell scripting. He has worked with global clients and in various industries, including IT, education, defense and space research, and the nonprofit sector. Follow him on Twitter, Facebook, Google+.

Share this on:

21 comment

  7. or try this…
    first make a bash script, ‘fixer.sh’


    #!/bin/bash
    replace CHANGEFROM CHANGETO $1.tmp
    rm $1
    mv $1.tmp $1

    now run this command line…

    $ grep CHANGEFROM |cut -d':' -f1 |xargs -n 1 fixer.sh

    the results is that all files in the directory (or whatever you grep for) will be changed automagically.

    just make sure the grep doesn’t include the fixer script itself, or it will die half-way through changing when execute permissions are reset!

    😉

    Reply Report comment

  9. Am I the only one who sees no difference in these?

    >> perl -p -i -e ’s/|00000000.00|/||/g’ myfile.txt
    >> I want to replace |00000000.00| with ||
    >> I get a compilation error.

    — —
    >> You need to write it as follows:
    >> perl -p -i -e ’s/|00000000.00|/||/g’ myfile.txt

    Reply Report comment

  13. `replace` can work with files, which may be simpler than writing a shell script as mentioned above. It will even convert files in place. Say you want to change an instance of ‘foo’ to ‘bar’ in all files in a certain directory, recursively. In bash,

    for i in `grep -lR foo dir/to/files`; do replace foo bar — $i; done

    Simply put, use two dashes to separate filenames from the from/to strings, and it’ll convert the files.

    Reply Report comment

  17. hi,
    thanks for the info..nice script. just wanted to add that when i used to ‘for’ loop version w/ ‘sed’ command (in Linux) it would read all the “*.sh” files that i wanted to substitue a string but did not actually make the change. just an fyi, i removed the single quote (‘) in the sed line and it works.

    See below:
    #!/bin/bash
    for f in /path/to/*.doc
    do
    sed -i s/UNIX/Linux/g “$f”
    done

    Thanks

    Reply Report comment

  20. Your example below will fail on filenames containing spaces due to the default internal field separator (IFS) breaking on a space character (default=$’ tn’):

    #!/bin/bash
for f in /path/to/*.txt
do
   replace UNIX Linux  "$f.new"
done

    To properly handle filenames with spaces, you should set IFS to break on new lines only:

    #!/bin/bash
oldIFS=IFS
IFS=$'n'
for f in /path/to/*.txt
do
   replace UNIX Linux  "$f.new"
done
IFS=oldIFS

    You can also simply run the command in a subshell by enclosing it in parenthesis and not have to worry about resetting IFS at the completion of the loop. Eg.:

    (IFS=$'n'; for f in /path/to/*.txt; do replace UNIX Linux  "$f.new"; done)

    However, while replace is a valid solution, I have found sed to be the most robust and much faster than perl for large search and replace tasks.

    Reply Report comment

Leave a Comment

Tagged as: Tags , , , , , , , , , , , , , ,