xargs: How To Control and Use Command Line Arguments

by on February 17, 2009 · 49 comments· LAST UPDATED September 2, 2010

in , ,

I am trying to use xargs command using shell pipes and not able to understand how to control and use command line arguments. For example I'd like to find out all *.c file located in 100s of sub-directories and move them to another directory called ~/old.src. How do I use command line args with xargs to achieve the same?

xargs command is designed to construct argument lists and invoke other utility. xargs reads items from the standard input or pipes, delimited by blanks or newlines, and executes the command one or more times with any initial-arguments followed by items read from standard input. Blank lines on the standard input are ignored.

xargs is more safer and easy to use

xargs functionality can be achived using the backquote feature of shell. But, it offers more options. It can deal with blanks or special characters in file names easily. It is often used with find, grep and other commands.

xargs examples

For example following example will print 1 2 3 4 using xargs (echo command is default)
$ echo 1 2 3 4 | xargs echo
OR
$ echo 1 2 3 4 | xargs
You can force xargs to use at most max-args arguments per command line. For example following will use first two argument per command:
$ echo 1 2 3 4 | xargs -n 2
Find all .bak files in or below the current directory and delete them.
$ find . -name "*.bak" -type f -print | xargs /bin/rm -f

{} as the argument list marker

{} is the default argument list marker. You need to use {} this with various command which take more than two arguments at a time. For example mv command need to know the file name. The following will find all .bak files in or below the current directory and move them to ~/.old.files directory:
$ find . -name "*.bak" -print0 | xargs -0 -I {} mv {} ~/old.files
You can rename {} to something else. In the following example {} is renamed as file. This is more readable as compare to previous example:
$ find . -name "*.bak" -print0 | xargs -0 -I file mv file ~/old.files
Where,

  1. -0 If there are blank spaces or characters (including newlines) many commands will not work. This option take cares of file names with blank space.
  2. -I Replace occurrences of replace-str in the initial-arguments with names read from standard input. Also, unquoted blanks do not terminate input items; instead the separator is the newline character.

Dealing file names with blank spaces and newline

The following will work incorrectly if there are any filenames containing newlines or spaces (it will find out all .mp3 file located in current directory and play them using mplayer):
$ find . -iname "*.mp3" -print | xargs mplayer
To get rid of this problem use -0 option:
$ find . -iname "*.mp3" -print0 | xargs -0 -I mp3file mplayer mp3file
To find out all *.c file located in 100s of subdirectories and move them to another directory called ~/old.src, use:
$ find /path/to/dir -iname "*.c" -print0 | xargs -0 -I file mv file ~/old.src

Avoiding errors and resource hungry problems with xargs and find combo

To copy all media files to another location called /bakup/iscsi, you can use cp as follows:
$ cp -r -v -p /share/media/mp3/ /backup/iscsi/mp3
However, cp command may fail if an error occurs such as if the number of files is too large for the cp command to handle. xargs in combination with find can handle such operation nicely. xargs is more resource efficient and will not halt with an error:

$ find /share/media/mp3/ -type f -name "*.mp3" -print0 | xargs -0 -r -I file cp -v -p file --target-directory=/bakup/iscsi/mp3

Please note that all of the above commands are tested with GNU/xargs version. BSD and UNIX xargs command may not have options such as -r. Please refer to your local xargs man page for further info:
man xargs

TwitterFacebookGoogle+PDF versionFound an error/typo on this page? Help us!

{ 49 comments… read them below or add one }

1 Juan February 17, 2009 at 7:40 pm

You can also do the following:

find /share/media/mp3/ -type f -name '*.mp3' -print -exec cp -v -p {} --target-directory=/bakup/iscsi/mp3 ";"

Reply

2 Ole Tange July 24, 2010 at 12:44 pm

If you know the file names do not contain \n and you have GNU Parallel http://www.gnu.org/software/parallel/ installed, then you can do:

find /share/media/mp3/ -type f -name ‘*.mp3′ -print | parallel -X cp -v -p {} /bakup/iscsi/mp3

It is somewhat shorter and will not spawn a cp for each copy. GNU Parallel also uses \n as its default separator which is helpful if you are using tools like head or tail.

Watch the intro video to GNU Parallel at http://www.youtube.com/watch?v=OpaiGYxkSuQ

Reply

3 sathiya February 18, 2009 at 11:04 am

nice reference to get to know about xargs.

Reply

4 dennytech February 18, 2009 at 1:11 pm

Pretty good. I’m a newbie and struggle but I got most of this. Thanks. My screen displayed zero as lower case o so the comment beginning “Where,” threw me for a minute. I searched the page for -(lower case o) and it wasn’t found, however, I did find -(numeral zero). Did “-print0″ refer to the blanks/newline problem or is it something else?

Reply

5 nixCraft February 18, 2009 at 2:50 pm

It is zero (0) and note small o letter.

The -print0 option prints ( displays ) the full file name on the screen, followed by a null character (instead of the newline character that -print uses). This allows file names that contain newlines or other types of white space to be correctly interpreted by programs that process the find output. This option corresponds to the -0 option of xargs.

HTH

Reply

6 yoander February 18, 2009 at 9:37 pm

Juan:
It’s true, you can get the same results using find -exec option but using xargs is more efficient

Reply

7 anil February 19, 2009 at 3:10 am

Dear
i am trying to find and delete some files but it gives error

~]# find . -name “abc*” xargs /bin/rm -f
find: paths must precede expression
Usage: find [path...] [expression]

~]ls abc abc1.c xargs /bin/rm abc
ls: xargs: No such file or directory
abc abc abc1.c /bin/rm

Can you suggest where i am going wrong.

Thanks
anil

Reply

8 nixCraft February 19, 2009 at 6:46 am

Try
find . -name "abc*" -print0| xargs /bin/rm -f

Reply

9 Mel February 24, 2009 at 4:38 am

Note that –target-directory is a GNU extension to cp(1).

Also, at least on FreeBSD:

find . -type f -exec grep -l '/bin/bash' {} +

is exactly the same as:

find . -type f -print0 |xargs -0 grep -l '/bin/bash'

The subtle difference being a terminating +, rather then a \;.

Last but not least, cp without shell globs doesn’t have a limit to the number of files it can handle. The problem comes when the shell converts the globs on the command line to actual arguments.

Reply

10 commbot March 24, 2009 at 11:42 pm

Thanks for your very useful and informative blog. I learn a lot from reading your tips and its has been a great deal of help for me on numerous occasions.

Using find xargs to change the UID or GID of files and directories fails if there is a space in the folder name. I tried two versions of the same command to accomplish this, with differing results. This is on FreeBSD 7.1

find -X / -group oldGID -print0 | xargs chgrp newGID
will work on filenames but not directories

find / -group oldGID -print0 | xargs -0 chgrp newGID
works for both

Reply

11 commbot March 24, 2009 at 11:49 pm

a mistake, first command should be:
find -X / -group oldGID -print | xargs chgrp newGID

Reply

12 Sam April 27, 2010 at 8:41 am

Your cp example can be done without the use of “-I file” and the “file” pattern to cp. The “-t” or “–target-directory” argument to cp allows the destination directory to be last on the command line.

-I has some other side-effects to xargs, and will prevent it from issuing as few cp commands as possible (use -p to see exactly what commands would result). With -I, if find returns 5 files, 5 cp commands will result.

This will have an identical result, but with fewer invocations of cp:

xargs -0 -r cp -v -p -t /bakup/iscsi/mp3

Of course, using find to execute the copy directly is also possible, as in the first comment, but this is a primer on the use of xargs — not every file list will be generated by find or findutils.

Reply

13 Sam April 27, 2010 at 8:43 am

Sorry, my first paragraph should read:

Your cp example can be done without the use of “-I file” and the “file” pattern to cp. The “-t” or “–target-directory” argument to cp allows multiple source files to be last on the command line.

Reply

14 Asrafi May 11, 2010 at 9:14 am

Dear Friends
I am facing a peculiar problem while trying to run a code with xargs. It runs successfully on cygwin, but can’t work on ubuntu. I’ve checked the xargs versions on both the platform, they are same. Can’t understand the problem really.

xargs -r -a input.txt -L1 code.exe > output.txt

This command simply take the inputs from input file on a line by line basis and after executing with the exe file writes the output in the output file.
in ubuntu it gives error
“xargs: tcas.exe: no such file or directory” .

Reply

15 hileon May 19, 2010 at 4:49 am

it seems this line :
$ find . -name “*.bak” -print0 | xargs -0 -I mv {} ~/old.files
should be:
$ find . -name “*.bak” -print0 | xargs -0 -I{} mv {} ~/old.files

nice post, very helpful to me.

Reply

16 max July 18, 2010 at 6:29 am

how do I pipe the john the ripper to hydra using xargs?
I tried but it doesn’t seem to work.

Reply

17 Nigel July 29, 2010 at 11:53 am

I have a bunch of script that cp files/dir from 1 place to another. The problem is that destination already has the same file albeit an older ones. I want cp to to replace them all without asking for user input. I try this on my fedora machine -
cp -rf /my/source/* /my/dest/ | xargs echo “yes”
cp still ends up needing me to enter “y” for this line -
cp: overwrite /my/dest/file1 ?

Is there some way to automate this copy/replace, without any user input?

thanks,

Reply

18 nixCraft July 29, 2010 at 12:49 pm
\cp -rf /my/source/* /my/dest/

OR

/bin/cp -rf /my/source/* /my/dest/

More info here

Reply

19 Nigel July 30, 2010 at 1:45 am

Thanks Vivek.

I should have been more clear. I need to run the scripts as root. And root’s bashrc alias cp to cp -i. Any calls to cp/rm/mv becomes interactive. I know that removing the alias will make cp work as usual. But its a shared machine, and i dont want to modify anything on it.

I need a scriptable way to echo yes to get around this. Any idea?

Reply

20 nixCraft July 30, 2010 at 2:47 am

FYI, when you use \cp or /bin/cp syntax you do not make any changes to root aliases or anything else. So they are safe to call from your script. But if you must, than use:

yes | cp -rf /my/source/* /my/dest/

Reply

21 Nigel July 30, 2010 at 3:57 am

Thanks Vivek, this was just what i was looking for.

Reply

22 Ole Tange September 1, 2010 at 3:00 pm

Give full path to cp:

/bin/cp -rf /my/source/* /my/dest/

Reply

23 Philippe Petrinko August 31, 2010 at 10:26 am

@Vivek,
There is 1 typo in “argument list marker” first example:
Just after the -I parameter, there should be “{}”
(unless this, the command fails)

Reply

24 felipe1982 September 1, 2010 at 1:36 pm

I’m trying to copy thousands of files to my Flash Drive using
`find /source-dir/ -type f -print0 | xargs -n 1000 cp -t /media/USB`
But this doesn’t stop at 1000 files. It continues. How can I force a maximum number of files, and quit when finished?
–felipe

Reply

25 nixCraft September 1, 2010 at 1:57 pm

No it will not work. If you pass 1000 arguments at a time then it will only work. You may need to use the for loop with counter using bash or any other scripting language :

#!/bin/bash
c=0
for f in /source-dir/*
do
  [ $c -eq 1000 ] && break
   /bin/cp -f "$f" /media/USB
  (( c++ ))
done

HTH

Reply

26 Ole Tange September 1, 2010 at 2:57 pm

If you have GNU Parallel http://www.gnu.org/software/parallel/ installed you can do this:

find /source-dir -type f | head -n 1000 | parallel cp {} /media/USB

It will work even if you have spaces and quotes in your filenames.

Watch the intro video for GNU Parallel to learn more:
http://www.youtube.com/watch?v=OpaiGYxkSuQ

Reply

27 Philippe Petrinko September 2, 2010 at 12:13 pm

@ felipe1982

I am sure you care for the resquest you posted,
By the way, would you give us some feedback,
for instance, did you try the code I suggested ?

I have tested it all right, so it should work for you.
Let us know.

Reply

28 Philippe Petrinko September 1, 2010 at 2:24 pm

Nice idea, Vivek.

Let me try this one-liner:

find /sourcedir -type f -print|head -n1000|while read; do echo -e -n "$REPLY\000"; done|xargs -0 -I {} cp "{}" /dest_dir

Limit is given by the head parameter,
but you need to rebuild a zero terminated list of arguments in order to allow xargs to use -0
(Zero terminated strings)

Reply

29 nixCraft September 1, 2010 at 5:31 pm

Philippe,

Done!

Also, I’m working on “preview button” mod. This will sort out this kind of issues. It is one of the most requested feature here. So by this weekend it should be up and running.

Reply

30 Philippe Petrinko September 1, 2010 at 7:01 pm

@Vivek
:-O You ‘ve got problems with Wordpress too. ;-)

The backslash is well positionned,
but there is only one zero after it, there should be three zeroes !
[again feel free to delete this post]

Reply

31 nixCraft September 2, 2010 at 11:03 am

Dam, wordpress always try to filter too much from comments to avoid html / js based attacks. BTW, I liked your idea.

Reply

32 Philippe Petrinko September 2, 2010 at 12:16 pm

@ Vivek

Still, you chapter ” {} as the argument list marker ”
still misses “{}” after the -I selector
into the first code example.

Would you at last modify your code ? ;-)

Reply

33 ryan December 3, 2010 at 9:13 pm

Uses available procs on a machine:
find / -type f -print0 | xargs -r0 -P$(nproc) -n10 md5sum # Process files in parallel over available processors

Reply

34 avatar_007 September 22, 2011 at 5:25 pm

xargs some how still dose not work for me
I do the following instead.
cat file_list.txt | awk ‘{print “cp “$0″ copyfiles_dir/”}’ | sh

Reply

35 tralev May 12, 2012 at 7:39 am

Nice hint. Thanks.

Reply

36 Gabriel Glusgold September 29, 2011 at 2:09 pm

How could I use xargs for removing several lines containing something?, example:

file1: contains
line1: eeee
line2: aaaa

file2: contains
line1: aaaa
line2: sssss
line3: nnnnn

then, The final objetive is make a command line using xarg like this:

grep -il “aaaa” file* | xarg grep -v “aaaa” _____here i dont know how is the followwing, I need something that eliminates “aaaa” from each file that find the “aaaa” line.

thanks in advance

Reply

37 Philippe Petrinko September 29, 2011 at 9:01 pm

Hi Gabriel.

Actually, there are many ways to achieve this task.

1) Let’s start by the simpliest one:

sed -i “/aaa/d” file*

Ok, this does not use [xargs]

2) Using [xargs]

grep -il ccc file*|xargs -I{} sed -i “/ccc/d” {}

Let us know some feedback.

– Philippe

Reply

38 Piyush February 7, 2012 at 1:57 pm

Hi all,
I wish to copy all files with extension jpg to a particular directory. To this effect, the command below will suffice
ls -tr | grep *.jpg| xargs -n1 -i cp {} ${tobe_copied_path1}/

Next, I wish to rename all ‘:’ in the file names to ‘.’ Will it be possible to do it using the same command line? I was trying something along the lines
ls -tr | grep *.jpg | xargs -n1 -i cp {} ${tobe_copied_path1}/`echo \{\} | sed ‘s/:/./g’`
but could not get through.
Any pointers on this one?
Thanks in advance.

Reply

39 Luis June 2, 2014 at 8:36 pm

Hi Piyush,

You could do this. For the sake of clarity, lets define some vars:

from_dir=from
to_dir=to
find $from_dir -name "*.jpg" -type f | xargs -n1 -I {} sh -c 'cp {} $0/`echo $(basename "{}") | sed s/:/./g`' $to_dir

The key is to invoke the shell from xargs.

Hope this help you.

Reply

40 Philippe Petrinko February 8, 2012 at 9:30 am

Hi Piyush,

First, use [find] to select files, not [ls]. Behave, try to follow Vivek’s advice! ;-)

Second, try to use bash shell variable pattern substitution
syntax is ${parameter/pattern/string}.

You’ll find what you need by reading the fantastic manual: type :

man bash

And look for pattern substitution. This will do the trick.
– Philippe

Reply

41 Philippe Petrinko February 8, 2012 at 11:45 am

Hi again, Piyush.

At this very moment, I did not find a trick with xargs to rename on the fly.
So, my solution is actually this one-liner command.
Assuming that source directory is named “from”, and destination is named “to”, you can tailor to your needs following compound command line:

fromdir=from; todir=to; find ${fromdir} -type f -iname “*.jpg” -print|while read; do f=${REPLY//:/.}; f=$(basename ${f}); cp $REPLY ${todir}/$f; done

Please let us know you feedback which would most certainly be instructive.

May be Vivek could find a solution using [xargs]?

– Philippe

Reply

42 Philippe Petrinko February 8, 2012 at 12:59 pm

At last, I got this working:
Trick is to build a command line with [echo] that is passed to bash through a pipe.
Again, you just have to change [f] and [t] variables with source and destination directories. That’s all folks!

export f=from; export t=to; find ${f} -type f -iname “*.jpg” -print0|xargs –null -n 1 -I{} echo ‘f1={}; f2=$(basename ${f1}); f2=${f2//:/.}; echo cp “${f1}” $t/”${f2}”‘|bash

Hope this helps.

Know a better way anyone?

(I used to do a similar trick some time ago, but do not remember yet… I think that was with some system call – almost similar)

– Philippe

Reply

43 Dinesh February 13, 2012 at 11:33 am

It’s really nice … Thank you

Reply

44 Biswojit Kar May 2, 2012 at 4:06 am

Hi,
How could I use mv command with xarg to rename all files with .txt extension to .tmp extension in all directories and subdirectories within current directory($HOME, .)??

Currently I can do the same using rename command as follows:
find . -name “*.txt” -print | xargs rename ‘s/.txt/.tmp/’
But I want to use mv instead of rename command to do the same. Please reply.
Thanks in advance!

Reply

45 andy September 9, 2012 at 1:25 pm

I CANNOT recommend xargs to be used with mplayer.

It’s maybe a bug in mplayer itself, as it will no longer allow you any interrupts per keyboard shortcuts (e. g ‘q’ for quit or ‘cursor right’ to ffwd in track).

It’s a very serious flaw.

-exec works fine instead. However, if you, for instance, want to have your find output sorted first ( | sort) you can’t use exec as exec will be executed first.

Reply

46 Philippe Petrinko September 9, 2012 at 10:12 pm

Hi Andy,

I can say I disagree about how serious the “flaw” may be, mostly because I would not find a useful case of using xargs for mplayer.

My point is: when I want mplayer to play a list of videos, I just use a playlist.
or, I use Menu [File] / [Open Folder], and I choose a folder where videos are.

Otherwise, yes, I think that if mplayer does not catch keystroke the right way. It gets interesting when things get weird, it is an opportunity to learn something new while investigating.

– Philippe

Reply

47 andy September 9, 2012 at 1:27 pm

Just testing, was this actually posted or is this in the queue for approval?
I’m confused now.

Reply

48 vivi March 12, 2013 at 4:41 pm

$ find . -name “*.bak” -print0 | xargs -0 -I file mv file ~/old.files

“old.files” here file is going to be replaced by the input item, i think it’s not we want

Reply

49 yogesh July 15, 2014 at 11:01 am

Awesome post! Keep it up Vivek babu! :)

Reply

Leave a Comment

Tagged as: , , , , , , , , , , , , , ,

Previous Faq:

Next Faq: