How to set rsync speed limit from eating all bandwidth with –bwlimit option

The rsync command is eating all my bandwidth from a backup script and generating tons of I/O. How do I set a limit for file transfer speed of rsync command on Linux or Unix-like system? How can I keep my rsync script from using all of my bandwidth?

rsync is a free and open-source command-line utility for efficiently transferring and synchronizing files between a computer and a hard drive. It also and across networked machines. You can easily limit I/O bandwidth using --bwlimit option. In this tutorial you will learn how to set up I/O limit using rsync command on Linux, MacOS, FreeBSD and Unix-like systems.
rsync-bwlimit-examples

ADVERTISEMENTS

Set rsync speed limit on Linux and Unix-like systems

  1. Open the Linux, Unix, or macOS terminal
  2. You can specify the maximum transfer rate for the data sent over the socket or network by typing the --bwlimit=RATE option to the NA command
  3. Set rsync bandwidth limit to 5000 KBytes per second: rsync --bwlimit=5000 source dest

How to keep rsync command from using all of my bandwidth

The syntax is:
rsync --bwlimit=KBPS src dst
rsync --bwlimit=KBPS [options] src dst
rsync --bwlimit=KBPS [options] src user@dst
rsync --bwlimit=KBPS [options] user@src /path/to/dir
rsync --bwlimit=KBPS -a -P /path/to/dir/ user@server1.cyberciti.biz

Examples

You set I/O limit in 1000 KBytes per second:
$ rsync --bwlimit=1000 /path/to/source /path/to/dest/
$ rsync --bwlimit=1000 /var/www/html/ \
backups@server1.cyberciti.biz:~/mysite.backups/

In this example, pull files from FreeNAS server with 5000KB/s in current directory:
$ rsync --bwlimit=5000 --delete -P -az -H --numeric-ids \
vivek@192.168.2.30:linode-www .

Understanding options

From the rsync man page:
The --bwlimit=KBPS option allows you to specify the maximum transfer rate. The RATE value can be suffixed with a string to indicate a size multiplier, and may be a fractional value (e.g. “–bwlimit=1.5m”). If no suffix is specified, the value will be assumed to be in units of 1024 bytes. For backward-compatibility reasons, the rate limit will be rounded to the nearest KiB unit, so no rate smaller than 1024 bytes per second is possible. Rsync writes data over the socket in blocks, and this option both limits the size of the blocks that rsync writes, and tries to keep the average transfer rate at the requested limit. Some “burstiness” may be seen where rsync writes out a block of data and then sleeps to bring the average rate into compliance.

Say hello to ionice

Use ionice command to set or get the I/O scheduling class and priority for a program such as rsync or your own backup script. So you can take control of I/O bandwidth using ionice utility on Linux as follows:
/usr/bin/ionice -c2 -n7 /root/scripts/nas.backup.full
OR
/usr/bin/ionice -c2 -n7 rsync \
-bwlimit=1000 /path/to/source /path/to/dest/

Where:

  • -c or --class {class} : Name or number of scheduling class, 0: none, 1: realtime, 2: best-effort, 3: idle.
  • -n or --classdata {num} : priority (0..7) in the specified scheduling class, only for the realtime and best-effort classes

How to limit rsync speed through trickle command option

Another option is to use trickle, It is a userspace bandwidth manager. The syntax is:
trickle -u uploadLimit program
trickle -d downloadLimit rsync
trickle -u {UPLOAD_LIMIT} -d {DOWNLOAD_LIMIT} program-binary

For instance,
trickle -s -d 3000 -u 3000 rsync /path/to/source/ /dest/
trickle -s -d 3000 -u 3000 rsync -avr /var/www/html/ vivek@server1.cyberciti.biz/var/www/html/

See the following tutorials for more info:

Conclusion

You learned how to set a rsync command speed limit using various techniques. The main advantage is to make sure that rsync does not eat up the network connection.

🐧 Get the latest tutorials on SysAdmin, Linux/Unix, Open Source/DevOps topics:
CategoryList of Unix and Linux commands
File Managementcat
FirewallAlpine Awall CentOS 8 OpenSUSE RHEL 8 Ubuntu 16.04 Ubuntu 18.04 Ubuntu 20.04
Network Utilitiesdig host ip nmap
OpenVPNCentOS 7 CentOS 8 Debian 10 Debian 8/9 Ubuntu 18.04 Ubuntu 20.04
Package Managerapk apt
Processes Managementbg chroot cron disown fg jobs killall kill pidof pstree pwdx time
Searchinggrep whereis which
User Informationgroups id lastcomm last lid/libuser-lid logname members users whoami who w
WireGuard VPNAlpine CentOS 8 Debian 10 Firewall Ubuntu 20.04

ADVERTISEMENTS
1 comment… add one

Leave a Reply

Your email address will not be published.

Use HTML <pre>...</pre>, <code>...</code> and <kbd>...</kbd> for code samples.