rsync is free and open source tool. It is useful to copy local or remote file. It reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination. You can use any one of the following options to add a total progress indicator when copying files from serverA to serverB or vice versa.
Tutorial details | |
---|---|
Difficulty | Easy (rss) |
Root privileges | No |
Requirements | None |
Time | 5m |
- rsync command to with --progress option.
- pv command – monitor the progress of data or data transfer through a pipe. This is a recommend option for most users.
rsync command to show progress bar
You need to use the --progress or -P option which show progress during file transfer. The syntax is as follows:
rsync --progress source dest rsync -P source dest rsync [options] --progress source dest rsync -av --progress /path/to/*.mp3 /nfs rsync -av --progress /path/to/*.avi -e user@remote.example.com:/data ############################################################### ## The following only works with the latest version of rsync ## ############################################################### rsync -av --info=progress2 source dest rsync -av --info=progress2 /path/to/*.avi -e user@remote.example.com:/data ## See the pv command below for more info & syntax ##
Examples
To copy file from /foo/*.tar.gz to a remote server called server1.cyberciti.biz, enter:
rsync -av -P /foo/*.tar.gz nixcraft@server1.cyberciti.biz:~
OR
rsync -av --progress /foo/*.tar.gz nixcraft@server1.cyberciti.biz:~
In this example, copy files from a remote server:
rsync -av --progress root@nas01:/tmp/*Office* .
Sample outputs:
receiving file list ... 2 files to consider MacOffice2011wSP2_English.dmg 1036923510 100% 14.72MB/s 0:01:07 (xfer#1, to-check=1/2) Office Mac Home and Student 2011 - 1PC-1User Keys.html 2230 100% 23.67kB/s 0:00:00 (xfer#2, to-check=0/2) sent 64 bytes received 1037052576 bytes 15363742.81 bytes/sec total size is 1036925740 speedup is 1.00
The --info=progress2 option shows statistics based on the whole transfer, rather than individual files. Use this flag without outputting a filename (e.g. avoid -v or specify –info=name0 if you want to see how the transfer is doing without scrolling the screen with a lot of names.
rsync -av --info=progress2 root@nas01:/tmp/*Office* .
Sample outputs:
sending incremental file list MacOffice2011wSP2_English.dmg 1,036,923,510 99% 39.90MB/s 0:00:24 (xfr#1, to-chk=0/2) sent 1,037,176,846 bytes received 35 bytes 40,673,603.18 bytes/sec total size is 1,036,925,740 speedup is 1.00
In this example copy from local computer to remote server:
$ rsync -v --progress debian-9-openstack-amd64.qcow2 vivek@centos7:~/virt/
Use pv command to monitor progress of rsync command
The pv command allows you to see the progress of data through a pipeline. It provides the following info:
- Time elapsed
- Percentage completed (with progress bar)
- Current throughput rate
- Total data transferred
- ETA
The syntax is:
rsync options source dest | pv -lpes Number-Of-Files
So if you have 42 files in /tmp/software and you would like to copy them to /nas10, enter:
rsync -vrltD --stats --human-readable /tmp/software /nas10 | pv -lep -s 42
OR
rsync -vrltD --stats --human-readable /tmp/software /nas10 | pv -lep -s 42 >/dev/null
Sample outputs:
See also
- Man pages: rsync(1)
🐧 9 comments so far... add one ↓
Category | List of Unix and Linux commands |
---|---|
File Management | cat |
Firewall | Alpine Awall • CentOS 8 • OpenSUSE • RHEL 8 • Ubuntu 16.04 • Ubuntu 18.04 • Ubuntu 20.04 |
Network Utilities | dig • host • ip • nmap |
OpenVPN | CentOS 7 • CentOS 8 • Debian 10 • Debian 8/9 • Ubuntu 18.04 • Ubuntu 20.04 |
Package Manager | apk • apt |
Processes Management | bg • chroot • cron • disown • fg • jobs • killall • kill • pidof • pstree • pwdx • time |
Searching | grep • whereis • which |
User Information | groups • id • lastcomm • last • lid/libuser-lid • logname • members • users • whoami • who • w |
WireGuard VPN | Alpine • CentOS 8 • Debian 10 • Firewall • Ubuntu 20.04 |
Also add -h to show human readable sizes (350M instead of 367071842).
um… That’s not a progress bar. That’s progress output.
--progress is what you are looking for. (incase anyone gets this in a google search like i did)
Thanks for the informative post.
If you are running an “update” to a large folder you may not know how many files will be updated in advance, thus making it difficult to supply the file count to PV without doing an rsync –dry-run first. You can capture the “update” file count into a variable like this:
Then you can call rsync and pv with a reference to the file count variable:
A few items to note:
–info=progress2 seems to be available only on 3.1.0+
I was using 3.0.9 and it complained no such option exists
It can be used with tar command too:
http://stackoverflow.com/questions/238073/how-to-add-a-progress-bar-to-a-shell-script
thank you and all the commenters for the different solutions. it works! :-)
Hi,
very usefull post.
Is there a way to do it also if i don’t know the number of files to copy/transfer?
How to do it all with onoe command?
Thanks
Already start your rsync and want to know what it’s working on?
lsof -ad3-999 -c rsync