Linux/UNIX: Argument list too long error for rm, cp, mv commands in a shell error

Author: Vivek Gite Last updated: January 30, 2015 14 comments

I‘m a new Linux user. We have a backup server and most our developers sftp or ftp to upload files. I just need to backup of last four weeks. If I try to list or remove directories I get an error cannot remove directory. I’m getting “Argument list too long” message. How do I solve this problem when executing commands in the bash/ksh/csh/tcsh/zsh shell?

Each command under Linux and UNIX-like systems accepts a parameter commonly known as command arguments (or args). For example, the command ‘cd /etc’ has one command line arguments, namely, /etc. Some command can accept more than two arguments and act on supplied args. For example, consider the following cp command:
cp /etc/file1 /etc/file2 /etc/file3 /mnt/pen[donotprint]

Tutorial details
Difficulty	Easy (rss)
Root privileges	No
Requirements	find & xargs commands
Time	N/A

[/donotprint]

Finding out maximum length of the command line argument

The maximum length of the command line argument is limited to *128k* within the Linux kernel. To find your limits, type:

getconf ARG_MAX

Sample outputs:

The limit affects the execve() kernel function, which is utilized by all other exec() functions (execl, execlp, execle, etc).

Option #1: How to get rid of “Argument list too long error” problem while using rm, ls, cp, mv or any other shell commands

The best way to deal with this problem is combination of the find and xargs commands. The syntax is:

find /dir/to/search -type f -name 'file-pattern' -print0 | xargs -0 -n 10 command-name
find /dir/to/search -type f -iname "*.EXT" -print0 | xargs -0 command-name
find /dir/to/search -type f -iname "*.EXT" -command

Just list files when you get “Argument list too long” error

Use any one of the following syntax to list only files

find /dir/to/search -type f -iname "*.EXT" -ls
find /nas01/data/ -type f -iname "*.doc" -ls

OR the following portable syntax:

find /nas01/data/ -name '*.doc' -exec ls {} +
## GNU/Find ##
find /nas01/data/ -type f -name '*.doc' -exec ls -l {} +

To avoid recursion use the following GNU find option:

find /nas01/data/ -name '*.doc' -maxdepth 1 -exec ls -l {} +

Just delete files when “Argument list too long” error

The syntax is as follows:

# GNU/find and BSD/find may not work on older *nix
find /dir/to/search -type f -iname "*.EXT" -delete
find ~/Downloads/ -type f -iname "*.avi" -delete
find ~/Projects/ -type f -iname "*.pdf" -delete
## use xargs and rm command ##
find ~/Downloads/ -name '*.avi' -print0 | xargs -0 -n 10 rm -f

More portable syntax:

find ~/Projects/ -name '*.pdf' -exec rm -vf {} +

To avoid recursion use the following GNU find option:

## GNU find and may work with bsd find too ##
find ~/Projects/ -name '*.pdf' -maxdepth 1 -exec rm -f {} +
find ~/Downloads/ -type f -iname '*.avi' -print0 | xargs -0 -n 10 rm -f

You can also use GNU parallel command:

## man parallel for more ##
find ~/Downloads/ -type f -iname '*.avi' -print0 | parallel -X -0 rm -vf

Where,

-name "*.pdf" : Search pattern.
-type f : Only match files.
-print 0 : Avoid nasty surprises when dealing with weird file names with blank spaces, ‘, “, \, tab, and newline characters. The -print0 primary in conjunction with xargs -0 as an effective alternative to dealing with weird filenames.
-maxdepth 1 : Descend at most one directory levels below the command line arguments i.e avoid recursion.
-exec rm -f {} + OR -exec ls {} + : Run rm or ls commands on as many as files possible.

Option #2: Iterate with the shell to avoid arg list too long error

The syntax is as follows and it is very slow and may not work at all:

for f in *; do command "$i"; done     
 
## delete all pdf files ##
for f in *.pdf; do /bin/rm -rf "$f"; done
 
############################################################################
### Delete all *.mp4 and *.avi files using pure Bash solution using arrays
### Based on http://mywiki.wooledge.org/BashFAQ/095 
############################################################################
 files=($HOME/Downloads/*.avi $HOME/Downloads/*.mp4)
 for ((i=0; i<${#files[*]}; i+=100)); do
   /bin/rm -vf "${files[@]:i:100}" 
 done

Command summary to to avoid the limit in a shell

## deals with weird filenames too ##
find /dir/to/search/ -name "file-pattern-search" -exec command {} + 
 
## find and xargs based solution ##
find /dir/to/search/ -name "file-pattern-search" -print0 | xargs -0 command
 
## For gnu parallel fans ##
find /dir/to/search/ -name 'file-pattern-search' -print0 | parallel -X -0 command
 
## older and not recommended solution; but kept here for the historical reasons only ##
for i in *; do command "$i"; done

See man pages for more info: mv(1)

🐧 If you liked this page, please support my work on Patreon or with a donation.
🐧 Get the latest tutorials on SysAdmin, Linux/Unix, Open Source/DevOps topics:

RSS feed or Weekly email newsletter
Share on Twitter • Facebook • 14 comments... add one ↓

UP NEXT

Category	List of Unix and Linux commands
File Management	cat
Firewall	Alpine Awall • CentOS 8 • OpenSUSE • RHEL 8 • Ubuntu 16.04 • Ubuntu 18.04 • Ubuntu 20.04
Network Utilities	dig • host • ip • nmap
OpenVPN	CentOS 7 • CentOS 8 • Debian 10 • Debian 8/9 • Ubuntu 18.04 • Ubuntu 20.04
Package Manager	apk • apt
Processes Management	bg • chroot • cron • disown • fg • jobs • killall • kill • pidof • pstree • pwdx • time
Searching	grep • whereis • which
User Information	groups • id • lastcomm • last • lid/libuser-lid • logname • members • users • whoami • who • w
WireGuard VPN	Alpine • CentOS 8 • Debian 10 • Firewall • Ubuntu 20.04