≡ Menu

xargs: How To Control and Use Command Line Arguments

I am trying to use xargs command using shell pipes and not able to understand how to control and use command line arguments. For example I'd like to find out all *.c file located in 100s of sub-directories and move them to another directory called ~/old.src. How do I use command line args with xargs to achieve the same?

xargs command is designed to construct argument lists and invoke other utility. xargs reads items from the standard input or pipes, delimited by blanks or newlines, and executes the command one or more times with any initial-arguments followed by items read from standard input. Blank lines on the standard input are ignored.

xargs is more safer and easy to use

xargs functionality can be achived using the backquote feature of shell. But, it offers more options. It can deal with blanks or special characters in file names easily. It is often used with find, grep and other commands.

xargs examples

For example following example will print 1 2 3 4 using xargs (echo command is default)
$ echo 1 2 3 4 | xargs echo
OR
$ echo 1 2 3 4 | xargs
You can force xargs to use at most max-args arguments per command line. For example following will use first two argument per command:
$ echo 1 2 3 4 | xargs -n 2
Find all .bak files in or below the current directory and delete them.
$ find . -name "*.bak" -type f -print | xargs /bin/rm -f

{} as the argument list marker

{} is the default argument list marker. You need to use {} this with various command which take more than two arguments at a time. For example mv command need to know the file name. The following will find all .bak files in or below the current directory and move them to ~/.old.files directory:
$ find . -name "*.bak" -print0 | xargs -0 -I {} mv {} ~/old.files
You can rename {} to something else. In the following example {} is renamed as file. This is more readable as compare to previous example:
$ find . -name "*.bak" -print0 | xargs -0 -I file mv file ~/old.files
Where,

  1. -0 If there are blank spaces or characters (including newlines) many commands will not work. This option take cares of file names with blank space.
  2. -I Replace occurrences of replace-str in the initial-arguments with names read from standard input. Also, unquoted blanks do not terminate input items; instead the separator is the newline character.

Dealing file names with blank spaces and newline

The following will work incorrectly if there are any filenames containing newlines or spaces (it will find out all .mp3 file located in current directory and play them using mplayer):
$ find . -iname "*.mp3" -print | xargs mplayer
To get rid of this problem use -0 option:
$ find . -iname "*.mp3" -print0 | xargs -0 -I mp3file mplayer mp3file
To find out all *.c file located in 100s of subdirectories and move them to another directory called ~/old.src, use:
$ find /path/to/dir -iname "*.c" -print0 | xargs -0 -I file mv file ~/old.src

Avoiding errors and resource hungry problems with xargs and find combo

To copy all media files to another location called /bakup/iscsi, you can use cp as follows:
$ cp -r -v -p /share/media/mp3/ /backup/iscsi/mp3
However, cp command may fail if an error occurs such as if the number of files is too large for the cp command to handle. xargs in combination with find can handle such operation nicely. xargs is more resource efficient and will not halt with an error:

$ find /share/media/mp3/ -type f -name "*.mp3" -print0 | xargs -0 -r -I file cp -v -p file --target-directory=/bakup/iscsi/mp3

Please note that all of the above commands are tested with GNU/xargs version. BSD and UNIX xargs command may not have options such as -r. Please refer to your local xargs man page for further info:
man xargs

Tweet itFacebook itGoogle+ itPDF itFound an error/typo on this page?

{ 51 comments… add one }

  • Juan February 17, 2009, 7:40 pm

    You can also do the following:

    find /share/media/mp3/ -type f -name '*.mp3' -print -exec cp -v -p {} --target-directory=/bakup/iscsi/mp3 ";"

    • Ole Tange July 24, 2010, 12:44 pm

      If you know the file names do not contain \n and you have GNU Parallel http://www.gnu.org/software/parallel/ installed, then you can do:

      find /share/media/mp3/ -type f -name ‘*.mp3’ -print | parallel -X cp -v -p {} /bakup/iscsi/mp3

      It is somewhat shorter and will not spawn a cp for each copy. GNU Parallel also uses \n as its default separator which is helpful if you are using tools like head or tail.

      Watch the intro video to GNU Parallel at http://www.youtube.com/watch?v=OpaiGYxkSuQ

  • sathiya February 18, 2009, 11:04 am

    nice reference to get to know about xargs.

  • dennytech February 18, 2009, 1:11 pm

    Pretty good. I’m a newbie and struggle but I got most of this. Thanks. My screen displayed zero as lower case o so the comment beginning “Where,” threw me for a minute. I searched the page for -(lower case o) and it wasn’t found, however, I did find -(numeral zero). Did “-print0” refer to the blanks/newline problem or is it something else?

    • nixCraft February 18, 2009, 2:50 pm

      It is zero (0) and note small o letter.

      The -print0 option prints ( displays ) the full file name on the screen, followed by a null character (instead of the newline character that -print uses). This allows file names that contain newlines or other types of white space to be correctly interpreted by programs that process the find output. This option corresponds to the -0 option of xargs.

      HTH

  • yoander February 18, 2009, 9:37 pm

    Juan:
    It’s true, you can get the same results using find -exec option but using xargs is more efficient

  • anil February 19, 2009, 3:10 am

    Dear
    i am trying to find and delete some files but it gives error

    ~]# find . -name “abc*” xargs /bin/rm -f
    find: paths must precede expression
    Usage: find [path…] [expression]

    ~]ls abc abc1.c xargs /bin/rm abc
    ls: xargs: No such file or directory
    abc abc abc1.c /bin/rm

    Can you suggest where i am going wrong.

    Thanks
    anil

    • nixCraft February 19, 2009, 6:46 am

      Try
      find . -name "abc*" -print0| xargs /bin/rm -f

  • Mel February 24, 2009, 4:38 am

    Note that –target-directory is a GNU extension to cp(1).

    Also, at least on FreeBSD:

    find . -type f -exec grep -l '/bin/bash' {} +

    is exactly the same as:

    find . -type f -print0 |xargs -0 grep -l '/bin/bash'

    The subtle difference being a terminating +, rather then a \;.

    Last but not least, cp without shell globs doesn’t have a limit to the number of files it can handle. The problem comes when the shell converts the globs on the command line to actual arguments.

  • commbot March 24, 2009, 11:42 pm

    Thanks for your very useful and informative blog. I learn a lot from reading your tips and its has been a great deal of help for me on numerous occasions.

    Using find xargs to change the UID or GID of files and directories fails if there is a space in the folder name. I tried two versions of the same command to accomplish this, with differing results. This is on FreeBSD 7.1

    find -X / -group oldGID -print0 | xargs chgrp newGID
    will work on filenames but not directories

    find / -group oldGID -print0 | xargs -0 chgrp newGID
    works for both

  • commbot March 24, 2009, 11:49 pm

    a mistake, first command should be:
    find -X / -group oldGID -print | xargs chgrp newGID

  • Sam April 27, 2010, 8:41 am

    Your cp example can be done without the use of “-I file” and the “file” pattern to cp. The “-t” or “–target-directory” argument to cp allows the destination directory to be last on the command line.

    -I has some other side-effects to xargs, and will prevent it from issuing as few cp commands as possible (use -p to see exactly what commands would result). With -I, if find returns 5 files, 5 cp commands will result.

    This will have an identical result, but with fewer invocations of cp:

    xargs -0 -r cp -v -p -t /bakup/iscsi/mp3

    Of course, using find to execute the copy directly is also possible, as in the first comment, but this is a primer on the use of xargs — not every file list will be generated by find or findutils.

  • Sam April 27, 2010, 8:43 am

    Sorry, my first paragraph should read:

    Your cp example can be done without the use of “-I file” and the “file” pattern to cp. The “-t” or “–target-directory” argument to cp allows multiple source files to be last on the command line.

  • Asrafi May 11, 2010, 9:14 am

    Dear Friends
    I am facing a peculiar problem while trying to run a code with xargs. It runs successfully on cygwin, but can’t work on ubuntu. I’ve checked the xargs versions on both the platform, they are same. Can’t understand the problem really.

    xargs -r -a input.txt -L1 code.exe > output.txt

    This command simply take the inputs from input file on a line by line basis and after executing with the exe file writes the output in the output file.
    in ubuntu it gives error
    “xargs: tcas.exe: no such file or directory” .

  • hileon May 19, 2010, 4:49 am

    it seems this line :
    $ find . -name “*.bak” -print0 | xargs -0 -I mv {} ~/old.files
    should be:
    $ find . -name “*.bak” -print0 | xargs -0 -I{} mv {} ~/old.files

    nice post, very helpful to me.

  • max July 18, 2010, 6:29 am

    how do I pipe the john the ripper to hydra using xargs?
    I tried but it doesn’t seem to work.

  • Nigel July 29, 2010, 11:53 am

    I have a bunch of script that cp files/dir from 1 place to another. The problem is that destination already has the same file albeit an older ones. I want cp to to replace them all without asking for user input. I try this on my fedora machine –
    cp -rf /my/source/* /my/dest/ | xargs echo “yes”
    cp still ends up needing me to enter “y” for this line –
    cp: overwrite /my/dest/file1 ?

    Is there some way to automate this copy/replace, without any user input?

    thanks,

    • nixCraft July 29, 2010, 12:49 pm
      \cp -rf /my/source/* /my/dest/

      OR

      /bin/cp -rf /my/source/* /my/dest/

      More info here

      • Nigel July 30, 2010, 1:45 am

        Thanks Vivek.

        I should have been more clear. I need to run the scripts as root. And root’s bashrc alias cp to cp -i. Any calls to cp/rm/mv becomes interactive. I know that removing the alias will make cp work as usual. But its a shared machine, and i dont want to modify anything on it.

        I need a scriptable way to echo yes to get around this. Any idea?

        • nixCraft July 30, 2010, 2:47 am

          FYI, when you use \cp or /bin/cp syntax you do not make any changes to root aliases or anything else. So they are safe to call from your script. But if you must, than use:

          yes | cp -rf /my/source/* /my/dest/
          
          • Nigel July 30, 2010, 3:57 am

            Thanks Vivek, this was just what i was looking for.

        • Ole Tange September 1, 2010, 3:00 pm

          Give full path to cp:

          /bin/cp -rf /my/source/* /my/dest/

  • Philippe Petrinko August 31, 2010, 10:26 am

    @Vivek,
    There is 1 typo in “argument list marker” first example:
    Just after the -I parameter, there should be “{}”
    (unless this, the command fails)

  • felipe1982 September 1, 2010, 1:36 pm

    I’m trying to copy thousands of files to my Flash Drive using
    `find /source-dir/ -type f -print0 | xargs -n 1000 cp -t /media/USB`
    But this doesn’t stop at 1000 files. It continues. How can I force a maximum number of files, and quit when finished?
    –felipe

    • nixCraft September 1, 2010, 1:57 pm

      No it will not work. If you pass 1000 arguments at a time then it will only work. You may need to use the for loop with counter using bash or any other scripting language :

      #!/bin/bash
      c=0
      for f in /source-dir/*
      do
        [ $c -eq 1000 ] && break
         /bin/cp -f "$f" /media/USB
        (( c++ ))
      done
      

      HTH

    • Ole Tange September 1, 2010, 2:57 pm

      If you have GNU Parallel http://www.gnu.org/software/parallel/ installed you can do this:

      find /source-dir -type f | head -n 1000 | parallel cp {} /media/USB

      It will work even if you have spaces and quotes in your filenames.

      Watch the intro video for GNU Parallel to learn more:
      http://www.youtube.com/watch?v=OpaiGYxkSuQ

    • Philippe Petrinko September 2, 2010, 12:13 pm

      @ felipe1982

      I am sure you care for the resquest you posted,
      By the way, would you give us some feedback,
      for instance, did you try the code I suggested ?

      I have tested it all right, so it should work for you.
      Let us know.

  • Philippe Petrinko September 1, 2010, 2:24 pm

    Nice idea, Vivek.

    Let me try this one-liner:

    find /sourcedir -type f -print|head -n1000|while read; do echo -e -n "$REPLY\000"; done|xargs -0 -I {} cp "{}" /dest_dir
    

    Limit is given by the head parameter,
    but you need to rebuild a zero terminated list of arguments in order to allow xargs to use -0
    (Zero terminated strings)

    • nixCraft September 1, 2010, 5:31 pm

      Philippe,

      Done!

      Also, I’m working on “preview button” mod. This will sort out this kind of issues. It is one of the most requested feature here. So by this weekend it should be up and running.

  • Philippe Petrinko September 1, 2010, 7:01 pm

    @Vivek
    :-O You ‘ve got problems with Wordpress too. ;-)

    The backslash is well positionned,
    but there is only one zero after it, there should be three zeroes !
    [again feel free to delete this post]

    • nixCraft September 2, 2010, 11:03 am

      Dam, wordpress always try to filter too much from comments to avoid html / js based attacks. BTW, I liked your idea.

  • Philippe Petrinko September 2, 2010, 12:16 pm

    @ Vivek

    Still, you chapter ” {} as the argument list marker ”
    still misses “{}” after the -I selector
    into the first code example.

    Would you at last modify your code ? ;-)

  • ryan December 3, 2010, 9:13 pm

    Uses available procs on a machine:
    find / -type f -print0 | xargs -r0 -P$(nproc) -n10 md5sum # Process files in parallel over available processors

  • avatar_007 September 22, 2011, 5:25 pm

    xargs some how still dose not work for me
    I do the following instead.
    cat file_list.txt | awk ‘{print “cp “$0″ copyfiles_dir/”}’ | sh

    • tralev May 12, 2012, 7:39 am

      Nice hint. Thanks.

  • Gabriel Glusgold September 29, 2011, 2:09 pm

    How could I use xargs for removing several lines containing something?, example:

    file1: contains
    line1: eeee
    line2: aaaa

    file2: contains
    line1: aaaa
    line2: sssss
    line3: nnnnn

    then, The final objetive is make a command line using xarg like this:

    grep -il “aaaa” file* | xarg grep -v “aaaa” _____here i dont know how is the followwing, I need something that eliminates “aaaa” from each file that find the “aaaa” line.

    thanks in advance

    • Philippe Petrinko September 29, 2011, 9:01 pm

      Hi Gabriel.

      Actually, there are many ways to achieve this task.

      1) Let’s start by the simpliest one:

      sed -i “/aaa/d” file*

      Ok, this does not use [xargs]

      2) Using [xargs]

      grep -il ccc file*|xargs -I{} sed -i “/ccc/d” {}

      Let us know some feedback.

      — Philippe

  • Piyush February 7, 2012, 1:57 pm

    Hi all,
    I wish to copy all files with extension jpg to a particular directory. To this effect, the command below will suffice
    ls -tr | grep *.jpg| xargs -n1 -i cp {} ${tobe_copied_path1}/

    Next, I wish to rename all ‘:’ in the file names to ‘.’ Will it be possible to do it using the same command line? I was trying something along the lines
    ls -tr | grep *.jpg | xargs -n1 -i cp {} ${tobe_copied_path1}/`echo \{\} | sed ‘s/:/./g’`
    but could not get through.
    Any pointers on this one?
    Thanks in advance.

    • Luis June 2, 2014, 8:36 pm

      Hi Piyush,

      You could do this. For the sake of clarity, lets define some vars:

      from_dir=from
      to_dir=to
      find $from_dir -name "*.jpg" -type f | xargs -n1 -I {} sh -c 'cp {} $0/`echo $(basename "{}") | sed s/:/./g`' $to_dir
      

      The key is to invoke the shell from xargs.

      Hope this help you.

  • Philippe Petrinko February 8, 2012, 9:30 am

    Hi Piyush,

    First, use [find] to select files, not [ls]. Behave, try to follow Vivek’s advice! ;-)

    Second, try to use bash shell variable pattern substitution
    syntax is ${parameter/pattern/string}.

    You’ll find what you need by reading the fantastic manual: type :

    man bash

    And look for pattern substitution. This will do the trick.
    — Philippe

    • Philippe Petrinko February 8, 2012, 11:45 am

      Hi again, Piyush.

      At this very moment, I did not find a trick with xargs to rename on the fly.
      So, my solution is actually this one-liner command.
      Assuming that source directory is named “from”, and destination is named “to”, you can tailor to your needs following compound command line:

      fromdir=from; todir=to; find ${fromdir} -type f -iname “*.jpg” -print|while read; do f=${REPLY//:/.}; f=$(basename ${f}); cp $REPLY ${todir}/$f; done

      Please let us know you feedback which would most certainly be instructive.

      May be Vivek could find a solution using [xargs]?

      — Philippe

      • Philippe Petrinko February 8, 2012, 12:59 pm

        At last, I got this working:
        Trick is to build a command line with [echo] that is passed to bash through a pipe.
        Again, you just have to change [f] and [t] variables with source and destination directories. That’s all folks!

        export f=from; export t=to; find ${f} -type f -iname “*.jpg” -print0|xargs –null -n 1 -I{} echo ‘f1={}; f2=$(basename ${f1}); f2=${f2//:/.}; echo cp “${f1}” $t/”${f2}”‘|bash

        Hope this helps.

        Know a better way anyone?

        (I used to do a similar trick some time ago, but do not remember yet… I think that was with some system call – almost similar)

        — Philippe

  • Dinesh February 13, 2012, 11:33 am

    It’s really nice … Thank you

  • Biswojit Kar May 2, 2012, 4:06 am

    Hi,
    How could I use mv command with xarg to rename all files with .txt extension to .tmp extension in all directories and subdirectories within current directory($HOME, .)??

    Currently I can do the same using rename command as follows:
    find . -name “*.txt” -print | xargs rename ‘s/.txt/.tmp/’
    But I want to use mv instead of rename command to do the same. Please reply.
    Thanks in advance!

  • andy September 9, 2012, 1:25 pm

    I CANNOT recommend xargs to be used with mplayer.

    It’s maybe a bug in mplayer itself, as it will no longer allow you any interrupts per keyboard shortcuts (e. g ‘q’ for quit or ‘cursor right’ to ffwd in track).

    It’s a very serious flaw.

    -exec works fine instead. However, if you, for instance, want to have your find output sorted first ( | sort) you can’t use exec as exec will be executed first.

    • Philippe Petrinko September 9, 2012, 10:12 pm

      Hi Andy,

      I can say I disagree about how serious the “flaw” may be, mostly because I would not find a useful case of using xargs for mplayer.

      My point is: when I want mplayer to play a list of videos, I just use a playlist.
      or, I use Menu [File] / [Open Folder], and I choose a folder where videos are.

      Otherwise, yes, I think that if mplayer does not catch keystroke the right way. It gets interesting when things get weird, it is an opportunity to learn something new while investigating.

      — Philippe

  • andy September 9, 2012, 1:27 pm

    Just testing, was this actually posted or is this in the queue for approval?
    I’m confused now.

  • vivi March 12, 2013, 4:41 pm

    $ find . -name “*.bak” -print0 | xargs -0 -I file mv file ~/old.files

    “old.files” here file is going to be replaced by the input item, i think it’s not we want

  • yogesh July 15, 2014, 11:01 am

    Awesome post! Keep it up Vivek babu! :)

  • Yordan Georgiev September 13, 2014, 8:40 am

    # handle space and other “difficult chars” in filenames with xarg
    cat $gen_include_file | perl -ne ‘s|\n|00|g;print’| xargs -0 -I “{}” zip -MM $zip_file “$component_name/$environment_name/{}”

  • Deevya Naresh December 22, 2014, 7:05 am

    I am not able to use it

    I am getting the below error when i tried to use it

    $ echo 1 2 3 4 | xargs echo
    -bash: $: command not found

    Kindly help

Leave a Comment