How to set rsync speed limit from eating all bandwidth with –bwlimit option

The rsync command is eating all my bandwidth from a backup script and generating tons of I/O. How do I set a limit for file transfer speed of rsync command on Linux or Unix-like system? How can I keep my rsync script from using all of my bandwidth?

rsync is a free and open-source command-line utility for efficiently transferring and synchronizing files between a computer and a hard drive. It also and across networked machines. You can easily limit I/O bandwidth using --bwlimit option. In this tutorial you will learn how to set up I/O limit using rsync command on Linux, MacOS, FreeBSD and Unix-like systems.

ADVERTISEMENTS

Set rsync speed limit on Linux and Unix-like systems

1. Open the Linux, Unix, or macOS terminal
2. You can specify the maximum transfer rate for the data sent over the socket or network by typing the --bwlimit=RATE option to the NA command
3. Set rsync bandwidth limit to 5000 KBytes per second: rsync --bwlimit=5000 source dest

How to keep rsync command from using all of my bandwidth

The syntax is:
rsync --bwlimit=KBPS src dst
rsync --bwlimit=KBPS [options] src dst
rsync --bwlimit=KBPS [options] src user@dst
rsync --bwlimit=KBPS [options] user@src /path/to/dir
rsync --bwlimit=KBPS -a -P /path/to/dir/ user@server1.cyberciti.biz

Examples

You set I/O limit in 1000 KBytes per second:
$ rsync --bwlimit=1000 /path/to/source /path/to/dest/
$ rsync --bwlimit=1000 /var/www/html/ \
backups@server1.cyberciti.biz:~/mysite.backups/
In this example, pull files from FreeNAS server with 5000KB/s in current directory:
$ rsync --bwlimit=5000 --delete -P -az -H --numeric-ids \
vivek@192.168.2.30:linode-www .

Understanding options

From the rsync man page:
The --bwlimit=KBPS option allows you to specify the maximum transfer rate. The RATE value can be suffixed with a string to indicate a size multiplier, and may be a fractional value (e.g. “–bwlimit=1.5m”). If no suffix is specified, the value will be assumed to be in units of 1024 bytes. For backward-compatibility reasons, the rate limit will be rounded to the nearest KiB unit, so no rate smaller than 1024 bytes per second is possible. Rsync writes data over the socket in blocks, and this option both limits the size of the blocks that rsync writes, and tries to keep the average transfer rate at the requested limit. Some “burstiness” may be seen where rsync writes out a block of data and then sleeps to bring the average rate into compliance.

Say hello to ionice

Use ionice command to set or get the I/O scheduling class and priority for a program such as rsync or your own backup script. So you can take control of I/O bandwidth using ionice utility on Linux as follows:
/usr/bin/ionice -c2 -n7 /root/scripts/nas.backup.full
OR
/usr/bin/ionice -c2 -n7 rsync \
-bwlimit=1000 /path/to/source /path/to/dest/
Where:

• -c or --class {class} : Name or number of scheduling class, 0: none, 1: realtime, 2: best-effort, 3: idle.
• -n or --classdata {num} : priority (0..7) in the specified scheduling class, only for the realtime and best-effort classes

How to limit rsync speed through trickle command option

Another option is to use trickle, It is a userspace bandwidth manager. The syntax is:
trickle -u uploadLimit program
trickle -d downloadLimit rsync
trickle -u {UPLOAD_LIMIT} -d {DOWNLOAD_LIMIT} program-binary
For instance,
trickle -s -d 3000 -u 3000 rsync /path/to/source/ /dest/
trickle -s -d 3000 -u 3000 rsync -avr /var/www/html/ vivek@server1.cyberciti.biz/var/www/html/
See the following tutorials for more info:

• Linux Limit A Specific User�s Shell Account Network Bandwidth Using Bash Shell and Trickle
• How to shape or restrict bandwidth under Linux / UNIX / BSD

Conclusion

You learned how to set a rsync command speed limit using various techniques. The main advantage is to make sure that rsync does not eat up the network connection.

Category	List of Unix and Linux commands
File Management	cat
Firewall	Alpine Awall • CentOS 8 • OpenSUSE • RHEL 8 • Ubuntu 16.04 • Ubuntu 18.04 • Ubuntu 20.04
Network Utilities	dig • host • ip • nmap
OpenVPN	CentOS 7 • CentOS 8 • Debian 10 • Debian 8/9 • Ubuntu 18.04 • Ubuntu 20.04
Package Manager	apk • apt
Processes Management	bg • chroot • cron • disown • fg • jobs • killall • kill • pidof • pstree • pwdx • time
Searching	grep • whereis • which
User Information	groups • id • lastcomm • last • lid/libuser-lid • logname • members • users • whoami • who • w
WireGuard VPN	Alpine • CentOS 8 • Debian 10 • Firewall • Ubuntu 20.04