I want to mirror a directory using ftp and want to exclude certain subdirectories (such as logs) and file ending with *.cvs. How do I exclude files while mirroring using lftp command under Linux or Unix like operating systems?


lftp ftp client has many features that makes it a better choice for this kind of works The mirror command can be used and specified source directory to local target directory. If target directory ends with a slash, the source base name is appended to target directory name. You can exclude files and sub-directories. You can also use glob character such as *.

lftp command exclude syntax

The syntax is as follows:

mirror --exclude RX 
mirror --exclude-glob GP

In this example exclude logs directory, enter:

mirror --exclude logs/ --exclude

You can use --exclude multiple times:

mirror --exclude logs/ --exclude reports/ --exclude-glob *.bak --exclude-glob *~$

The following will exclude all *.cvs files:

mirror --exclude-glob *.cvs

Putting it all together

Create a file called mirror.http.lftp as follows:

set ftp:list-options -a
set cmd:fail-exit true
open ftp.cyberciti.biz.biz:/var/www/
lcd /nas06/backup/cyberciti.biz/
mirror --delete --exclude logs/ --exclude tmp/ --exclude-glob *~$
quit

Create or edit an authentication file called ~/.netrc – this file contains configuration and autologin information for the File Transfer Protocol client ftp and lftp:

machine ftp.cyberciti.biz
login MyUserNameHer
password myPassWordHere

Save and close the file. Run the job as follows:
lftp -f mirror.http.lftp
OR
lftp -f mirror.http.lftp > ~/mirror.http.lftp.log