$ scp -r ~/projects/ user@hostB:/home/delta/projects/
How do I tell scp command to exclude particular file or directory at the Linux/Unix command line?
One can use scp command to securely copy files between hosts on a network. It uses ssh for data transfer and authentication purpose. Typical scp command syntax is as follows:
scp file1 user@host:/path/to/dest/
scp -r /path/to/source/ user@host:/path/to/dest/
scp [options] /dir/to/source/ user@host:/dir/to/dest/
Tutorial requirements | |
---|---|
Operating system/app | macOS/Linux/Unix-like OS |
Root privileges required | No |
Difficulty | Easy (rss) |
Estimated completion time | 3m |
Scp exclude files syntax and alternative
I don’t think so you can filter or exclude files when using scp command. However, there is a great workaround to exclude files and copy it securely using ssh. This page explains how to filter or excludes files when using scp to copy a directory recursively.
How to use rsync command to exclude files
The syntax is:
rsync -av -e ssh --exclude='*.out' /path/to/source/ user@hostB:/path/to/dest/
Where,
- -a : Recurse into directories i.e. copy all files and subdirectories. Also, turn on archive mode and all other options (-rlptgoD)
- -v : Verbose output
- -e ssh : Use ssh for remote shell so everything gets encrypted
- --exclude='*.out' : exclude files matching PATTERN e.g. *.out or *.c and so on.
Example of rsync command
In this example copy all file recursively from ~/virt/ directory but exclude all *.new files:
$ rsync -av -e ssh --exclude='*.new' ~/virt/ root@centos7:/tmp
Sample outputs:
scp exclude files with extglob bash option
The rsync command will fail if the rsync not found on the remote server. In that case try the following scp command that uses bash shell pattern matching in the current directory (it won’t work with the -r option):
$ ls
Sample outputs:
centos71.log centos71.qcow2 centos71.qcow2.new centos71.v2.qcow2.new meta-data user-data
Copy everything in the current directory except .new file(s):
$ shopt -s extglob
$ scp !(*.new) root@centos7:/tmp/
Sample outputs:
centos71.log 100% 4262 1.3MB/s 00:00 centos71.qcow2 100% 836MB 32.7MB/s 00:25 meta-data 100% 47 18.5KB/s 00:00 user-data 100% 1543 569.7KB/s 00:00
Understanding extglob shell option to exclude files with scp and rsync
If the extglob bash shell option is enabled using the shopt command (builtin), several extended pattern matching operators are recognized by the Linux system as follows:
Operators | Description |
---|---|
?(pattern-list) | Matches zero or one occurrence of the given patterns |
*(pattern-list) | Matches zero or more occurrences of the given patterns |
+(pattern-list) | Matches one or more occurrences of the given patterns |
@(pattern-list) | Matches one of the given patterns |
!(pattern-list) | Matches anything except one of the given patterns |
Conclusion
This page showed you how to exclude files when copying files between hosts on a network under Linux or Unix-like operating systems. Pretty useful to filter out files when using the scp command/rsync command to copy directory recursively. See the following man pages for more information:
$ man 1 rsync
$ man bash
$ man 1 scp
🐧 Get the latest tutorials on Linux, Open Source & DevOps via:
- RSS feed or Weekly email newsletter
- Share on Twitter • Facebook • 1 comment... add one ↓
Category | List of Unix and Linux commands |
---|---|
File Management | cat |
Firewall | Alpine Awall • CentOS 8 • OpenSUSE • RHEL 8 • Ubuntu 16.04 • Ubuntu 18.04 • Ubuntu 20.04 |
Network Utilities | dig • host • ip • nmap |
OpenVPN | CentOS 7 • CentOS 8 • Debian 10 • Debian 8/9 • Ubuntu 18.04 • Ubuntu 20.04 |
Package Manager | apk • apt |
Processes Management | bg • chroot • cron • disown • fg • jobs • killall • kill • pidof • pstree • pwdx • time |
Searching | grep • whereis • which |
User Information | groups • id • lastcomm • last • lid/libuser-lid • logname • members • users • whoami • who • w |
WireGuard VPN | Alpine • CentOS 8 • Debian 10 • Firewall • Ubuntu 20.04 |
there is a typo
$ scp !(.new)* root@centos7:/tmp/ #match files who don’t begin by `.new`
$ scp !(*.new) root@centos7:/tmp/