lftp is a file transfer program. It allows sophisticated FTP, HTTP/HTTPS, and other connections. If the site URL is specified, then lftp will connect to that site otherwise a connection has to be established with the open command. It is an essential tool for all a Linux/Unix command line users. I have already written about Linux ultra fast command line download accelerator such as Axel and prozilla. lftp is another tool for the same job with more features. lftp can handle seven file access methods:
So what is unique about lftp?
- Every operation in lftp is reliable, that is any not fatal error is ignored, and the operation is repeated. So if downloading breaks, it will be restarted from the point automatically. Even if FTP server does not support REST command, lftp will try to retrieve the file from the very beginning until the file is transferred completely.
- lftp has shell-like command syntax allowing you to launch several commands in parallel in the background.
- lftp has a builtin mirror which can download or update a whole directory tree. There is also a reverse mirror (mirror -R) which uploads or updates a directory tree on the server. The mirror can also synchronize directories between two remote servers, using FXP if available.
How to use lftp as download accelerator
lftp has pget command. It allows you download files in parallel. The syntax is
lftp -e 'pget -n NUM -c url; exit'
For example, download http://kernel.org/pub/linux/kernel/v2.6/linux-220.127.116.11.tar.bz2 file using pget in 5 parts:
$ cd /tmp
$ lftp -e 'pget -n 5 -c http://kernel.org/pub/linux/kernel/v2.6/linux-18.104.22.168.tar.bz2'
45108964 bytes transferred in 57 seconds (775.3K/s) lftp :~>quit
- pget – Download files in parallel
- -n 5 – Set maximum number of connections to 5
- -c – Continue broken transfer if lfile.lftp-pget-status exists in the current directory
How to use lftp to accelerate ftp/https download on Linux/Unix
Another try with added exit command:
$ lftp -e 'pget -n 10 -c https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.15.tar.xz; exit'
HTML 5 Video 01: lftp command in action
A note about parallel downloading
Please note that by using download accelerator you are going to put a load on remote host. Also note that lftp may not work with sites that do not support multi-source downloads or blocks such requests at firewall level.NA command offers many other features. Refer to lftp man page for more information:
🐧 7 comments so far... add one ↓
|Category||List of Unix and Linux commands|
|Disk space analyzers||ncdu • pydf|
|Firewall||Alpine Awall • CentOS 8 • OpenSUSE • RHEL 8 • Ubuntu 16.04 • Ubuntu 18.04 • Ubuntu 20.04|
|Network Utilities||NetHogs • dig • host • ip • nmap|
|OpenVPN||CentOS 7 • CentOS 8 • Debian 10 • Debian 8/9 • Ubuntu 18.04 • Ubuntu 20.04|
|Package Manager||apk • apt|
|Processes Management||bg • chroot • cron • disown • fg • jobs • killall • kill • pidof • pstree • pwdx • time|
|Searching||grep • whereis • which|
|User Information||groups • id • lastcomm • last • lid/libuser-lid • logname • members • users • whoami • who • w|
|WireGuard VPN||Alpine • CentOS 8 • Debian 10 • Firewall • Ubuntu 20.04|