I want to mirror a directory using ftp and want to exclude certain subdirectories (such as logs) and file ending with *.cvs. How do I exclude files while mirroring using lftp command under Linux or Unix like operating systems?
lftp ftp client has many features that makes it a better choice for this kind of works The mirror command can be used and specified source directory to local target directory. If target directory ends with a slash, the source base name is appended to target directory name. You can exclude files and sub-directories. You can also use glob character such as *.
lftp command exclude syntax
The syntax is as follows:
mirror --exclude RX mirror --exclude-glob GP |
In this example exclude logs directory, enter:
mirror --exclude logs/ --exclude |
You can use --exclude multiple times:
mirror --exclude logs/ --exclude reports/ --exclude-glob *.bak --exclude-glob *~$ |
The following will exclude all *.cvs files:
mirror --exclude-glob *.cvs |
Putting it all together
Create a file called mirror.http.lftp as follows:
set ftp:list-options -a set cmd:fail-exit true open ftp.cyberciti.biz.biz:/var/www/ lcd /nas06/backup/cyberciti.biz/ mirror --delete --exclude logs/ --exclude tmp/ --exclude-glob *~$ quit |
Create or edit an authentication file called ~/.netrc – this file contains configuration and autologin information for the File Transfer Protocol client ftp and lftp:
machine ftp.cyberciti.biz login MyUserNameHer password myPassWordHere |
Save and close the file. Run the job as follows:
lftp -f mirror.http.lftp
OR
lftp -f mirror.http.lftp > ~/mirror.http.lftp.log