How to set rsync speed limit from eating all bandwidth with -bwlimit option

The rsync command is eating all my bandwidth from a backup script and generating tons of I/O. How do I set a limit for file transfer speed of rsync command on Linux or Unix-like system? How can I keep my rsync script from using all of my bandwidth?

rsync is a free and open-source command-line utility for efficiently transferring and synchronizing files between a computer and a hard drive. It also and across networked machines. You can easily limit I/O bandwidth using --bwlimit option. In this tutorial you will learn how to set up I/O limit using rsync command on Linux, MacOS, FreeBSD and Unix-like systems.
rsync-bwlimit-examples

Set rsync speed limit on Linux and Unix-like systems

Open the Linux, Unix, or macOS terminal
You can specify the maximum transfer rate for the data sent over the socket or network by typing the --bwlimit=RATE option to the NA command
Set rsync bandwidth limit to 5000 KBytes per second: rsync --bwlimit=5000 source dest

How to keep rsync command from using all of my bandwidth

The syntax is:
rsync --bwlimit=KBPS src dst rsync --bwlimit=KBPS [options] src dst rsync --bwlimit=KBPS [options] src user@dst rsync --bwlimit=KBPS [options] user@src /path/to/dir rsync --bwlimit=KBPS -a -P /path/to/dir/ user@server1.cyberciti.biz

Examples

You set I/O limit in 1000 KBytes per second:
$ rsync --bwlimit=1000 /path/to/source /path/to/dest/ $ rsync --bwlimit=1000 /var/www/html/ \ backups@server1.cyberciti.biz:~/mysite.backups/
In this example, pull files from FreeNAS server with 5000KB/s in current directory:
$ rsync --bwlimit=5000 --delete -P -az -H --numeric-ids \ vivek@192.168.2.30:linode-www .

Understanding options

From the rsync man page:
The --bwlimit=KBPS option allows you to specify the maximum transfer rate. The RATE value can be suffixed with a string to indicate a size multiplier, and may be a fractional value (e.g. “–bwlimit=1.5m”). If no suffix is specified, the value will be assumed to be in units of 1024 bytes. For backward-compatibility reasons, the rate limit will be rounded to the nearest KiB unit, so no rate smaller than 1024 bytes per second is possible. Rsync writes data over the socket in blocks, and this option both limits the size of the blocks that rsync writes, and tries to keep the average transfer rate at the requested limit. Some “burstiness” may be seen where rsync writes out a block of data and then sleeps to bring the average rate into compliance.

Say hello to ionice

Use ionice command to set or get the I/O scheduling class and priority for a program such as rsync or your own backup script. So you can take control of I/O bandwidth using ionice utility on Linux as follows:
/usr/bin/ionice -c2 -n7 /root/scripts/nas.backup.full
OR
/usr/bin/ionice -c2 -n7 rsync \ -bwlimit=1000 /path/to/source /path/to/dest/
Where:

-c or --class {class} : Name or number of scheduling class, 0: none, 1: realtime, 2: best-effort, 3: idle.
-n or --classdata {num} : priority (0..7) in the specified scheduling class, only for the realtime and best-effort classes

How to limit rsync speed through trickle command option

Another option is to use trickle, It is a userspace bandwidth manager. The syntax is:
trickle -u uploadLimit program trickle -d downloadLimit rsync trickle -u {UPLOAD_LIMIT} -d {DOWNLOAD_LIMIT} program-binary
For instance,
trickle -s -d 3000 -u 3000 rsync /path/to/source/ /dest/ trickle -s -d 3000 -u 3000 rsync -avr /var/www/html/ vivek@server1.cyberciti.biz/var/www/html/
See the following tutorials for more info:

Conclusion

You learned how to set a rsync command speed limit using various techniques. The main advantage is to make sure that rsync does not eat up the network connection.

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin, DevOps engineer, and a trainer for the Linux operating system/Unix shell scripting. Get the latest tutorials on SysAdmin, Linux/Unix and open source topics via RSS/XML feed or weekly email newsletter.