rsync is a free and open-source command-line utility for efficiently transferring and synchronizing files between a computer and a hard drive. It also and across networked machines. You can easily limit I/O bandwidth using --bwlimit option. In this tutorial you will learn how to set up I/O limit using rsync command on Linux, MacOS, FreeBSD and Unix-like systems.
Set rsync speed limit on Linux and Unix-like systems
- Open the Linux, Unix, or macOS terminal
- You can specify the maximum transfer rate for the data sent over the socket or network by typing the --bwlimit=RATE option to the NA command
- Set rsync bandwidth limit to 5000 KBytes per second: rsync --bwlimit=5000 source dest
How to keep rsync command from using all of my bandwidth
The syntax is:
rsync --bwlimit=KBPS src dst
rsync --bwlimit=KBPS [options] src dst
rsync --bwlimit=KBPS [options] src user@dst
rsync --bwlimit=KBPS [options] user@src /path/to/dir
rsync --bwlimit=KBPS -a -P /path/to/dir/ user@server1.cyberciti.biz
Examples
You set I/O limit in 1000 KBytes per second:
$ rsync --bwlimit=1000 /path/to/source /path/to/dest/
$ rsync --bwlimit=1000 /var/www/html/ \
backups@server1.cyberciti.biz:~/mysite.backups/
In this example, pull files from FreeNAS server with 5000KB/s in current directory:
$ rsync --bwlimit=5000 --delete -P -az -H --numeric-ids \
vivek@192.168.2.30:linode-www .
Understanding options
From the rsync man page:
The --bwlimit=KBPS option allows you to specify the maximum transfer rate. The RATE value can be suffixed with a string to indicate a size multiplier, and may be a fractional value (e.g. “–bwlimit=1.5m”). If no suffix is specified, the value will be assumed to be in units of 1024 bytes. For backward-compatibility reasons, the rate limit will be rounded to the nearest KiB unit, so no rate smaller than 1024 bytes per second is possible. Rsync writes data over the socket in blocks, and this option both limits the size of the blocks that rsync writes, and tries to keep the average transfer rate at the requested limit. Some “burstiness” may be seen where rsync writes out a block of data and then sleeps to bring the average rate into compliance.
Say hello to ionice
Use ionice command to set or get the I/O scheduling class and priority for a program such as rsync or your own backup script. So you can take control of I/O bandwidth using ionice utility on Linux as follows:
/usr/bin/ionice -c2 -n7 /root/scripts/nas.backup.full
OR
/usr/bin/ionice -c2 -n7 rsync \
-bwlimit=1000 /path/to/source /path/to/dest/
Where:
- -c or --class {class} : Name or number of scheduling class, 0: none, 1: realtime, 2: best-effort, 3: idle.
- -n or --classdata {num} : priority (0..7) in the specified scheduling class, only for the realtime and best-effort classes
How to limit rsync speed through trickle command option
Another option is to use trickle, It is a userspace bandwidth manager. The syntax is:
trickle -u uploadLimit program
trickle -d downloadLimit rsync
trickle -u {UPLOAD_LIMIT} -d {DOWNLOAD_LIMIT} program-binary
For instance,
trickle -s -d 3000 -u 3000 rsync /path/to/source/ /dest/
trickle -s -d 3000 -u 3000 rsync -avr /var/www/html/ vivek@server1.cyberciti.biz/var/www/html/
See the following tutorials for more info:
- Linux Limit A Specific Users Shell Account Network Bandwidth Using Bash Shell and Trickle
- How to shape or restrict bandwidth under Linux / UNIX / BSD
Conclusion
You learned how to set a rsync command speed limit using various techniques. The main advantage is to make sure that rsync does not eat up the network connection.
ð§ 1 comment so far... add one ↓
Category | List of Unix and Linux commands |
---|---|
File Management | cat |
Firewall | Alpine Awall • CentOS 8 • OpenSUSE • RHEL 8 • Ubuntu 16.04 • Ubuntu 18.04 • Ubuntu 20.04 |
Network Utilities | dig • host • ip • nmap |
OpenVPN | CentOS 7 • CentOS 8 • Debian 10 • Debian 8/9 • Ubuntu 18.04 • Ubuntu 20.04 |
Package Manager | apk • apt |
Processes Management | bg • chroot • cron • disown • fg • jobs • killall • kill • pidof • pstree • pwdx • time |
Searching | grep • whereis • which |
User Information | groups • id • lastcomm • last • lid/libuser-lid • logname • members • users • whoami • who • w |
WireGuard VPN | Alpine • CentOS 8 • Debian 10 • Firewall • Ubuntu 20.04 |
Nice article!
Or install FireQOS: https://github.com/firehol/netdata/wiki/You-should-install-QoS-on-all-your-servers