Howto: Use wget Recursively Download All FTP Directories

I would like to copy all of my files and directories from UNIX server to Linux workstation. How do I use wget command to recursively download whole FTP directories stored at /home/tom/ from ftp.example.com to local directory called /home/tom/backup?

GNU Wget is a free Linux / UNIX utility for non-interactive download of files from the Web or and FTP servers, as well as retrieval through HTTP proxies. GNU/wget has been designed for robustness over slow dialup internet or unstable network connections. If a download fails due to a network problem, it will keep retrying until the whole file has been retrieved. If the server supports regetting, it will instruct the server to continue the download from where it left off.

ADVERTISEMENTS

wget Recursive Example

You can use the -r ( recursive retrieving ) option as follows. You can also pass your ftp username and password to the wget command. First, make a backup directory in your $HOME directory:

mkdir ~/backup/
cd ~/backup/

Now, use wget command as follows:

wget -r ftp://username:password@ftp.example.com/
wget -r ftp://tom:myPassword@ftp.example.com/home/tom/
wget -r ftp://tom:myPassword@ftp.example.com/var/www/

wget recursive ftp with mirroring option

The -m option turns on mirroring i.e. it turns on recursion and time-stamping, sets infinite recursion depth and keeps FTP directory listings:

wget -m ftp://username:Password@ftp.example.com/
wget -m ftp://username:Password@ftp.example.com/var/www/html
🐧 Get the latest tutorials on SysAdmin, Linux/Unix, Open Source/DevOps topics:
CategoryList of Unix and Linux commands
File Managementcat
FirewallCentOS 8 OpenSUSE RHEL 8 Ubuntu 16.04 Ubuntu 18.04 Ubuntu 20.04
Network Utilitiesdig host ip nmap
OpenVPNCentOS 7 CentOS 8 Debian 10 Debian 8/9 Ubuntu 18.04 Ubuntu 20.04
Package Managerapk apt
Processes Managementbg chroot cron disown fg jobs killall kill pidof pstree pwdx time
Searchinggrep whereis which
User Informationgroups id lastcomm last lid/libuser-lid logname members users whoami who w
WireGuard VPNCentOS 8 Debian 10 Firewall Ubuntu 20.04

ADVERTISEMENTS
4 comments… add one
  • Ravi Jan 25, 2013 @ 22:22

    the mirroring option is not retaining the time stamp of directories but only of the files in those directories…can you think of a solution where someone would want to retain the directory structure of the same.

  • disney May 22, 2013 @ 2:04

    it assumes the current directory as root so we dont need to specify, but I have a parent directory in parallel to root as var
    the below only works and pulls data from root/var/tests and not /var/tests
    wget64 -r ftp://root:root@192.168.0.1/var/log/msm8960
    any suggestions?

  • disney May 22, 2013 @ 2:07

    sorry my pwd is root, so how do I change that for wget64?

  • Jason Huebel Dec 23, 2014 @ 19:57

    I also like to drop the directories that it will normally create when using this command. So, starting with the examples above:

    wget -r ftp://username:password@ftp.example.com/home/username

    This command would create a local directory structure that looks like this:

    /ftp.example.com/home/username/

    So, I like to add the following command-line options:

    wget -r -nH –cut-dirs=2 ftp://username:password@ftp.example.com/home/username

    -nH removes the http://ftp.example.com root directory from the local copy. –cut-dirs=2 removes /home/username from the local copy. So you end up with everything being saved to the current directory (and recursively copying folders from the remote /home/username folder.

Leave a Reply

Your email address will not be published.

Use HTML <pre>...</pre>, <code>...</code> and <kbd>...</kbd> for code samples.