How do I find duplicate files in a given set of directories and delete them using a shell script or a command line options? How do I get rid of double duplicates files stored in ~/foo and /u2/foo directory?
You need to use a tool called fdupes. It will searche the given path for duplicate files. Such files are found by comparing file sizes and MD5 signatures, followed by a byte-by-byte comparison. fdupes is a nice tool to get rid of duplicate files.
Type the following command under Debian / Ubuntu Linux:
# apt-get install fdupes
Type the following command under Redhat / RHEL / Fedota / CentOS Linux, enter (turn on rpmforge repo before running the following yum command):
# yum install fdupes
How Do I Use fdupes?
Find duplicate files in /etc/ directory, enter:
# fdupes /etc
How Do I Delete Unwanted Files?
You can force fdupes to prompt you for files to preserve, deleting all others (use this with care otherwise you may loss data):
# fdupes -d /etc
 /etc/vimrc  /etc/virc Set 1 of 1, preserve files [1 - 2, all]: 1 [+] /etc/vimrc [-] /etc/virc
How Do Recursively Search Directory?
You can recursively search every directory given follow subdirectories encountered within the -r option, enter:
# fdupes -r /dir1
How Do I Find Dupes In Two Directories?
Type the command as follows:
# fdupes /dir1 /dir2
# fdupes -r /etc /data/etc /nas95/etc
How Do I See Size Of Duplicate Files?
Type the following command with the -S option:
# fdupes -S /etc
1533 bytes each: /etc/vimrc /etc/virc
- man page fdupes
- 30 Cool Open Source Software I Discovered in 2013
- 30 Handy Bash Shell Aliases For Linux / Unix / Mac OS X
- Top 30 Nmap Command Examples For Sys/Network Admins
- 25 PHP Security Best Practices For Sys Admins
- 20 Linux System Monitoring Tools Every SysAdmin Should Know
- 20 Linux Server Hardening Security Tips
- Linux: 20 Iptables Examples For New SysAdmins
- Top 20 OpenSSH Server Best Security Practices
- Top 20 Nginx WebServer Best Security Practices
- 20 Examples: Make Sure Unix / Linux Configuration Files Are Free From Syntax Errors
- 15 Greatest Open Source Terminal Applications Of 2012
- My 10 UNIX Command Line Mistakes
- Top 10 Open Source Web-Based Project Management Software
- Top 5 Email Client For Linux, Mac OS X, and Windows Users
- The Novice Guide To Buying A Linux Laptop