Use wget Command To Download Files From HTTPS Domains

How do I download a file using https://example.com/dl/foo.tar.gz using wget command line utility?

GNU Wget is a free utility for the non-interactive download of files from the Web. It supports various protocols such as HTTP, HTTPS, and FTP protocols and retrieval through HTTP proxies. Wget is non-interactive, meaning that it can work in the background while the user is not logged on to the system. A perfect tool for your shell scripts to grab files from HTTPS enabled website too.
Tutorial details
Difficulty level Easy
Root privileges No
Requirements wget on Linux or Unix-like systems
Est. reading time 2m

Using wget command to download files from HTTPS websites

The syntax is:
wget https://cyberciti.biz/foo/bar.tar.gz
wget [options] https://url/file

If you don’t want about checking the validity of the certificate, just pass the option --no-check-certificate to the wget command-line:
wget --no-check-certificate https://cyberciti.biz/foo/bar.tar.gz
You can pass the --no-proxy option to the wget command. This option tells wget not to use proxies, even if the appropriate ‘*_proxy‘ environment variable is defined:
wget --no-proxy https://cyberciti.biz/foo/bar.tar.gz
Set the https proxy using https_proxy environment variable:

export https_proxy="https://USERNAME-HERE:PASSWORD-HERE@Server-Name-Here:PORT-NUMBER-HERE/"
export https_proxy="https://vivek:TopSecretePassword@server1.cyberciti.biz:3128/"
wget https://cyberciti.biz/foo/bar.tar.gz

Examples

Here is what we see:

export http_proxy="10.8.0.1:3128"
export https_proxy="10.8.0.1:3128"
wget 'https://www.cyberciti.biz/files/memems/images/TCP-IP%20Meme.png'
ls -l *.png
xdg-open TCP*.png

Use wget Command To Download Files From HTTPS Domains with Proxy server
We can pass proxy server username and password to the server using the wget command as follows:

export https_proxy="https://server1.cyberciti.biz:3128/"
wget --http-user="USER" --http-password="PASSWORD" https://cyberciti.biz/foo/bar.tar.gz</kbd>

ftp_proxy shell environment variable

The ftp_proxy variable should contain the URL of the proxy for FTP connections. It is quite common that http_proxy and ftp_proxy are set to the same URL. Hence:
export http_proxy="10.8.0.1:3128"
export https_proxy="${http_proxy}"
export ftp_proxy="${http_proxy}"
wget https://url
wget ftp://url

We can set and specify the username user and password for an FTP server when using wget using the --ftp-user=user and --ftp-password=password option:

wget --ftp-user=vivek \
--ftp-password=secrete \
ftp://ftp.server1.cybercit.biz/foo.pdf

Without an FTP username and password, wget will use the anonymous FTP option. We can force TLS/SSL FTP (FTPS) as follows for security and privacy reasons:

wget ftps://url/file.foo
wget --ftps-implicit ftps://url/file.foo
wget --ftps-implicit \
-ftp-user=nixcraft \
--ftp-password=my_password_here \
--no-check-certificate \
ftps://url/file.foo

Please note that FTPS consists of initializing SSL/TLS from the very beginning of the control connection. This option does not send an “AUTH TLS” command. It assumes the server speaks FTPS and directly starts an SSL/TLS connection.

Conclusion

The GNU wget has many more options. I suggest you read the wget man page by typing the following Linux or Unix/macOS command:
man wget
wget --help


🐧 Get the latest tutorials on Linux, Open Source & DevOps via RSS feed or Weekly email newsletter.

🐧 5 comments so far... add one

CategoryList of Unix and Linux commands
Disk space analyzersdf ncdu pydf
File Managementcat tree
FirewallAlpine Awall CentOS 8 OpenSUSE RHEL 8 Ubuntu 16.04 Ubuntu 18.04 Ubuntu 20.04
Network UtilitiesNetHogs dig host ip nmap
OpenVPNCentOS 7 CentOS 8 Debian 10 Debian 8/9 Ubuntu 18.04 Ubuntu 20.04
Package Managerapk apt
Processes Managementbg chroot cron disown fg jobs killall kill pidof pstree pwdx time
Searchinggrep whereis which
User Informationgroups id lastcomm last lid/libuser-lid logname members users whoami who w
WireGuard VPNAlpine CentOS 8 Debian 10 Firewall Ubuntu 20.04
5 comments… add one
  • Ahmed Dev Aug 16, 2014 @ 1:13

    Thanks That really helps

  • JZA Nov 13, 2014 @ 3:14

    I want to know how can I set by default the –no-check-certificate on wget for https. Is this possible?

    • samy Dec 11, 2014 @ 20:23

      Not sure if you found the answer in the meantime?
      You need to use: check-certificate = off (or check_certificate = off) in your .wgetrc startup file.

  • Ravinder Sep 7, 2017 @ 19:53

    Hi,

    I have used wget to login to the application with save cookies parameter in the first step. I have used .pem files for the SSL handshake.
    I pass userid and password to login to the app.

    In the 2nd step I use the same certificate .pem files , also the same cookies file generated in 1st step. This step actually calls the main function of the application to perform certain operation/job.

    Now the client doesn’t want to give userid and pass in the 1st step, I am not able to find a way how to directly invoke the function of 2nd step to call the application to do the job.

    Can you tell me how wget can be used in windows to call the main function by directly logging in to application.

    Thanks
    Ravi

  • Aneesh Sep 14, 2017 @ 14:32

    I am receiving ” Unsupported scheme” while using https://

Leave a Reply

Your email address will not be published.

Use HTML <pre>...</pre> for code samples. Still have questions? Post it on our forum