Each command under Linux and UNIX-like systems accepts a parameter commonly known as command arguments (or args). For example, the command ‘cd /etc’ has one command line arguments, namely, /etc. Some command can accept more than two arguments and act on supplied args. For example, consider the following cp command:
cp /etc/file1 /etc/file2 /etc/file3 /mnt/pen[donotprint]
| Tutorial details | |
|---|---|
| Difficulty | Easy (rss) |
| Root privileges | No |
| Requirements | find & xargs commands |
| Time | N/A |
The cp command has total four command line arguments. The shell can hold a maximum of 131072 bytes for command line arguments. If you try to pass more than that number you will greeted with an error that read as follows:
Argument list too long
Remember each character requires one byte of storage.
Finding out maximum length of the command line argument
The maximum length of the command line argument is limited to *128k* within the Linux kernel. To find your limits, type:
getconf ARG_MAX
Sample outputs:
2621440
The limit affects the execve() kernel function, which is utilized by all other exec() functions (execl, execlp, execle, etc).
Option #1: How to get rid of “Argument list too long error” problem while using rm, ls, cp, mv or any other shell commands
The best way to deal with this problem is combination of the find and xargs commands. The syntax is:
find /dir/to/search -type f -name 'file-pattern' -print0 | xargs -0 -n 10 command-name find /dir/to/search -type f -iname "*.EXT" -print0 | xargs -0 command-name find /dir/to/search -type f -iname "*.EXT" -command
Just list files when you get “Argument list too long” error
Use any one of the following syntax to list only files
find /dir/to/search -type f -iname "*.EXT" -ls find /nas01/data/ -type f -iname "*.doc" -ls
OR the following portable syntax:
find /nas01/data/ -name '*.doc' -exec ls {} + ## GNU/Find ## find /nas01/data/ -type f -name '*.doc' -exec ls -l {} +
To avoid recursion use the following GNU find option:
find /nas01/data/ -name '*.doc' -maxdepth 1 -exec ls -l {} +
Just delete files when “Argument list too long” error
The syntax is as follows:
# GNU/find and BSD/find may not work on older *nix find /dir/to/search -type f -iname "*.EXT" -delete find ~/Downloads/ -type f -iname "*.avi" -delete find ~/Projects/ -type f -iname "*.pdf" -delete ## use xargs and rm command ## find ~/Downloads/ -name '*.avi' -print0 | xargs -0 -n 10 rm -f
More portable syntax:
find ~/Projects/ -name '*.pdf' -exec rm -vf {} +
To avoid recursion use the following GNU find option:
## GNU find and may work with bsd find too ## find ~/Projects/ -name '*.pdf' -maxdepth 1 -exec rm -f {} + find ~/Downloads/ -type f -iname '*.avi' -print0 | xargs -0 -n 10 rm -f
You can also use GNU parallel command:
## man parallel for more ## find ~/Downloads/ -type f -iname '*.avi' -print0 | parallel -X -0 rm -vf
Where,
- -name "*.pdf" : Search pattern.
- -type f : Only match files.
- -print 0 : Avoid nasty surprises when dealing with weird file names with blank spaces, ‘, “, \, tab, and newline characters. The -print0 primary in conjunction with xargs -0 as an effective alternative to dealing with weird filenames.
- -maxdepth 1 : Descend at most one directory levels below the command line arguments i.e avoid recursion.
- -exec rm -f {} + OR -exec ls {} + : Run rm or ls commands on as many as files possible.
Option #2: Iterate with the shell to avoid arg list too long error
The syntax is as follows and it is very slow and may not work at all:
for f in *; do command "$i"; done ## delete all pdf files ## for f in *.pdf; do /bin/rm -rf "$f"; done ############################################################################ ### Delete all *.mp4 and *.avi files using pure Bash solution using arrays ### Based on http://mywiki.wooledge.org/BashFAQ/095 ############################################################################ files=($HOME/Downloads/*.avi $HOME/Downloads/*.mp4) for ((i=0; i<${#files[*]}; i+=100)); do /bin/rm -vf "${files[@]:i:100}" done
Command summary to to avoid the limit in a shell
## deals with weird filenames too ## find /dir/to/search/ -name "file-pattern-search" -exec command {} + ## find and xargs based solution ## find /dir/to/search/ -name "file-pattern-search" -print0 | xargs -0 command ## For gnu parallel fans ## find /dir/to/search/ -name 'file-pattern-search' -print0 | parallel -X -0 command ## older and not recommended solution; but kept here for the historical reasons only ## for i in *; do command "$i"; done
See man pages for more info: mv(1)
🐧 Get the latest tutorials on Linux, Open Source & DevOps via RSS feed or Weekly email newsletter.
🐧 14 comments so far... add one ↓
🐧 14 comments so far... add one ↓
Related Tutorials
| Category | List of Unix and Linux commands |
|---|---|
| File Management | cat |
| Firewall | Alpine Awall • CentOS 8 • OpenSUSE • RHEL 8 • Ubuntu 16.04 • Ubuntu 18.04 • Ubuntu 20.04 |
| Network Utilities | dig • host • ip • nmap |
| OpenVPN | CentOS 7 • CentOS 8 • Debian 10 • Debian 8/9 • Ubuntu 18.04 • Ubuntu 20.04 |
| Package Manager | apk • apt |
| Processes Management | bg • chroot • cron • disown • fg • jobs • killall • kill • pidof • pstree • pwdx • time |
| Searching | grep • whereis • which |
| User Information | groups • id • lastcomm • last • lid/libuser-lid • logname • members • users • whoami • who • w |
| WireGuard VPN | Alpine • CentOS 8 • Debian 10 • Firewall • Ubuntu 20.04 |
How to find out who all user modified which files
how to allow telnet or ssh login from a specified ip to freebsd server and other to be denied
steps of Maintainence of a freebsd server and what all pre precaution need to be taken
What about for recursive copy??
just use:
ls|xargs rm
cheers
tom
Tom you genious! Gone through all the complex methods that didn’t work, and you use the most simple way. Thank you.
-bash: /bin/mv: Argument list too long
I am Getting th eavove error message while moving file from one dir to other.please help me to solve this problem.I hv root permission too.
How about this:
find . | xargs rm
It will wipe out just about anything.
ViVEk GITe, not agree with you. The wilcards in any shell enviroments are replaced before execute the comand line. For example:
$ touch a1 a2 a3 a4 a5 a6
$ ls a*
a1 a2 a3 a4 a5 a6
Will work fine, because the ‘a*’ is replaced before executing the ls command but the parameters list is not long enough. But, creating for example a extend list of files (10000 empty files), the ‘ls’ command will show you how the parameters list is too long even when you write ‘a*’ as the only parameter!!
See:
$ i=1
$ while [ $i -le 10000 ]; do
> touch a$i
> i=$((i+1))
> done
$ ls | wc -l
10000
$ ls a*
ksh: /usr/bin/ls: 0403-027 The parameter list is too long.
That is because the wilcard is replaced before executing ls + arguments command.
In that case I agree with Tom solution… ;)
Thank you for this!
Btw, your iterating example is missing a y
(which for very lazy copypasta addicts like myself is dangerous)
@Vivek
Right, this topic should be modified, at least for two reasons:
1. Using wildcards does not solve anything. It is just false.
2. The more appropriate way, is to follow Unix philosophy, that is, do one thing and pass it to another program: => use xargs !
https://en.wikipedia.org/wiki/Unix_philosophy
Please modifiy this old 2007 topic that is just outdated and does not show how good you are at *nix! You are much better than that! (At least you are now ;-) )
— Philippe
You know me well.. as usual thanks a million.
Whaoo!
Congratulations for your quick reaction and full rewriting of this topic.
Just to be thorough, I would clearly advice __not__ to use the [-exec] option of find, but use find to pipe to xargs in any situation.
There are deep and strong reasons why, again, it is more appropriate to not use [-exec], let [find] do one thing, and then pipe to [xargs].
I could elaborate, and share this with you if you want, feel free to e-mail me in private if you need to.
Many thanks again for your professional work here, helping us using *nix !
— Philippe
The path you entered, is too long. Enter a shorter path
File Name could not be found. Check the spelling of the filename,
and verify that the file location is correct.