How Do I Replace A Text String In Many Files At Once?

last updated in Categories Linux, Shell scripting, Tips, UNIX

The replace command is a string-replacement utility. It changes strings in place in files or on the standard input. This command uses a finite state machine to match longer strings first. It can be used to swap strings. This command is similar to the Perl -pie syntax or sed (stream editor) command.

Please note that the replace command is part of is MySQL database system. If you don’t have MySQL installed, you don’t have replace command.

Advertisements

Syntax

replace OLD-STRING NEW-STRING < INPUT-FILE > OUTPUT-FILE

Examples

To replace all occurrences of word UNIX with Linux, enter:
$ replace UNIX Linux < oldfile > newfile

The replace command can be used in a pipeline, run:
$ cat /etc/passwd | replace : '|'

You can skip the cat command, enter:
$ replace : '|' < /etc/passwd

It also supports few special characters in string replacement:

  • \^ : Match start of line.
  • $ : Match end of line.

How Do I Update All *.txt Files At Once?

You use bash for loop as follows:

#!/bin/bash
for f in /path/to/*.txt
do
   replace UNIX Linux < "$f" > "$f.new"
done

The replace command does not understand regular expression. To use regular expression try the sed command or Perl.

Sed Command Example

To replace all occurrences of word UNIX with Linux using the sed command, enter:

sed 's/UNIX/Linux/g' < input.file > output.file

OR

sed -i 's/UNIX/Linux/g' input.file

OR use bash shell for loop as follows to update all *.doc files at once:

#!/bin/bash
for f in /path/to/*.doc
do
   sed -i  's/UNIX/Linux/g' "$f"
done

Updated for accuracy!

ADVERTISEMENTS

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin, DevOps engineer, and a trainer for the Linux operating system/Unix shell scripting. Get the latest tutorials on SysAdmin, Linux/Unix and open source topics via RSS/XML feed or weekly email newsletter.

21 comment

  4. what about this line:

    perl -p -i -e ‘s/|00000000.00|/||/g’ myfile.txt

    I want to replace |00000000.00| with ||

    I get a compilation error.

  5. You need to write it as follows:

    perl -p -i -e ‘s/|00000000.00|/||/g’ myfile.txt

    || got some special meaning (regex) | will disable it

  7. or try this…
    first make a bash script, ‘fixer.sh’


    #!/bin/bash
    replace CHANGEFROM CHANGETO $1.tmp
    rm $1
    mv $1.tmp $1

    now run this command line…

    $ grep CHANGEFROM |cut -d':' -f1 |xargs -n 1 fixer.sh

    the results is that all files in the directory (or whatever you grep for) will be changed automagically.

    just make sure the grep doesn’t include the fixer script itself, or it will die half-way through changing when execute permissions are reset!

    😉

  9. Am I the only one who sees no difference in these?

    >> perl -p -i -e ’s/|00000000.00|/||/g’ myfile.txt
    >> I want to replace |00000000.00| with ||
    >> I get a compilation error.

    — —
    >> You need to write it as follows:
    >> perl -p -i -e ’s/|00000000.00|/||/g’ myfile.txt

  13. `replace` can work with files, which may be simpler than writing a shell script as mentioned above. It will even convert files in place. Say you want to change an instance of ‘foo’ to ‘bar’ in all files in a certain directory, recursively. In bash,

    for i in `grep -lR foo dir/to/files`; do replace foo bar — $i; done

    Simply put, use two dashes to separate filenames from the from/to strings, and it’ll convert the files.

  14. I couldn’t even find this command in Ubunto 10.04 (or in repositories). I ended up using rpl instead. Is really fun for databending images.

  16. I want to replace a string in sub directories also(in Linux), is there any specific command for that?

  17. hi,
    thanks for the info..nice script. just wanted to add that when i used to ‘for’ loop version w/ ‘sed’ command (in Linux) it would read all the “*.sh” files that i wanted to substitue a string but did not actually make the change. just an fyi, i removed the single quote (‘) in the sed line and it works.

    See below:
    #!/bin/bash
    for f in /path/to/*.doc
    do
    sed -i s/UNIX/Linux/g “$f”
    done

    Thanks

  19. I noticed in my bash shell I need to do the following on my Mac:

    sed -i ” ‘s/UNIX/Linux/g’ input.file

    Otherwise I get stdin error.

  20. Your example below will fail on filenames containing spaces due to the default internal field separator (IFS) breaking on a space character (default=$’ tn’):

    #!/bin/bash
for f in /path/to/*.txt
do
   replace UNIX Linux  "$f.new"
done

    To properly handle filenames with spaces, you should set IFS to break on new lines only:

    #!/bin/bash
oldIFS=IFS
IFS=$'n'
for f in /path/to/*.txt
do
   replace UNIX Linux  "$f.new"
done
IFS=oldIFS

    You can also simply run the command in a subshell by enclosing it in parenthesis and not have to worry about resetting IFS at the completion of the loop. Eg.:

    (IFS=$'n'; for f in /path/to/*.txt; do replace UNIX Linux  "$f.new"; done)

    However, while replace is a valid solution, I have found sed to be the most robust and much faster than perl for large search and replace tasks.

    Have a question? Post it on our forum!