lftp to accelerate ftp / http download speed under Linux and UNIX

by on August 10, 2007 · 7 comments· LAST UPDATED August 10, 2007

in , ,

lftp is a file transfer program that allows sophisticated ftp, http and other connections to other hosts. If site is specified then lftp will connect to that site otherwise a connection has to be established with the open command. This is an essential tool for all a Linux admin. I've already written about Linux ultra fast command line download accelerator such as Axel and prozilla. lftp is another tool for same job with more features.

lftp can handle seven file access methods :
=> ftp
=> ftps
=> http
=> https
=> hftp
=> fish
=> sftp
=> file

So what's special about lftp?

  • Every operation in lftp is reliable, that is any not fatal error is ignored and the operation is repeated. So if downloading breaks, it will be restarted from the
    point automatically. Even if ftp server does not support REST command, lftp will try to retrieve the file from the very beginning until the file is transferred completely.
  • lftp has shell-like command syntax allowing you to launch several commands in parallel in background
  • lftp has builtin mirror which can download or update a whole directory tree. There is also reverse mirror (mirror -R) which uploads or updates a directory tree on server. Mirror can also synchronize directories between two remote servers, using FXP if available.

Use lftp as download accelerator

lftp has pget command. It allows you download files in parallel. lftp pget command syntax:
lftp -e 'pget -n 5 ftp://path/to/file'

For example, download http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.22.2.tar.bz2 file using pget in 5 parts:
$ cd /tmp
$ lftp -e 'pget -n 5 http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.22.2.tar.bz2'

Output:

45108964 bytes transferred in 57 seconds (775.3K/s)
lftp :~>quit

A note about parallel downloading

Please note that by using download accelerator you are going to put a load on remote host. Also note that lftp may not work with sites that do not support multi-source downloads or blocks such requests at firewall level.

lftp command offers many other features. Refer to lftp man page for more information:
man lftp

TwitterFacebookGoogle+PDF versionFound an error/typo on this page? Help us!

{ 7 comments… read them below or add one }

1 Terry Rankine November 8, 2007 at 4:20 am

Since we are all lazy people, put the quit in the -e command.


lftp -e 'pget -n 5 http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.22.2.tar.bz2;quit'

Reply

2 nixCraft November 8, 2007 at 1:09 pm

Terry,

Heh nice, I just forgot to add quit.

Reply

3 rotyyu January 20, 2009 at 3:44 am

This tools is really crazy……..

Reply

4 John Schultz August 19, 2009 at 5:57 pm

I was trying to use the lftp mirror command to copy a website from a Windows server to my Ubuntu desktop. The directories are copied just fine, but not the files in the directories. Am I missing something or is lftp not compatible with Windows servers?

Any answers will be most appreciated.

John

Reply

5 Martin C. September 10, 2012 at 4:50 pm

Is there any way to use lftp to download two or more files at a time (parallel downloading)?

Reply

6 linuxicle June 18, 2013 at 1:31 pm

It’s been a good help for geeker

Reply

7 Oleg September 10, 2014 at 8:51 pm

Please let me know how to download lftp to UNIX?

Reply

Leave a Comment

Tagged as: , ,

Previous post:

Next post: