How To Use wget With Username and Password for FTP / HTTP File Retrieval

last updated in Categories , , , , ,

How do I use GNU wget FTP or HTTP client tool to download files from password protected web pages on Linux or Unix-like system? Is there a way to download a file using username and password from a config file?

The gnu wget command supports username and password combo for both FTP and HTTP file retrieval.

Syntax

The syntax is:
wget options url
wget --user=NAME --password='PASSWORD' url
wget --user=NAME --password='PASSWORD' ftp://url/path/file.name
wget --user=NAME --password='PASSWORD' http://url/path/file.name

Where,

  1. --user=userNameHere: Your FTP/HTTP username.
  2. --password=passWordHere: Your HTTP/FTP password.

Specify the username user and password password for both FTP and HTTP file retrieval. These parameters can be overridden using the --ftp-user and --ftp-password options for FTP connections and the --http-user and --http-password options for HTTP connections. To download a file called foo.pdf from theos.in domain, enter:
$ wget --user=vivek --password='myPassword' http://theos.in/protected/area/foo.pdf
OR
wget --user=vivek --ask-password http://192.168.1.10/docs/foo.pdf
Sample outputs:

Fig.01: wget command in action with HTTP auth.
Fig.01: wget command in action with HTTP auth.

Ask for a password

The passwords on the command line are visible using ps command. This is a security feature. Pass the --ask-password option to pompt for a password for each connection established. Cannot be specified when --password is being used, because they are mutually exclusive:
$ wget --user=vivek --ask-password http://theos.in/protected/area/foo.pdf

How do I read username and password from a file?

This is a security feature. Create a ~/.wgetrc file. The syntax is as follows to store login credentials:

$ cat ~/.wgetrc
user=vivek
password=myCoolPasswordhere

Of course, you can configure separate FTP and HTTP credentials as follows too:

$ cat ~/.wgetrc
ftp_user=vivekftp
ftp_password=myCoolFTPPassword
http_user=vivekhttp
http_password=myCoolHTTPPassword

Make sure only you can read the ~/.wgetrc file:
$ chmod 0400 ~/.wgetrc
$ chown vivek:vivek ~/.wgetrc
Please note that storing unencrypted passwords is also considered a security risk.

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin, DevOps engineer, and a trainer for the Linux operating system/Unix shell scripting. Get the latest tutorials on SysAdmin, Linux/Unix and open source topics via RSS/XML feed or weekly email newsletter.

31 comment

  7. I need to download a file from my server on another machine with wget, using -p password as an argument is not an option since people could simply find my password with a ps command! Do you know anyway I could force wget to ask for my password after typing the command? like when do mysql -u root -p and it will ask you for the root password?
    thanks

  9. wget does not work directly if the site redirects to form based login page, i.e. no HTTP 401 returned

  11. As easy at that! This made downloading a large XML file much easier than using a browser which ended up crashing 9 out of 10 times.

    Many thanks.

    Chris.

  12. I tried the following command which gave a Error Response
    The Response is
    Resolving cambuild.qalab.cam.novell.com… 164.99.129.35
    Connecting to cambuild.qalab.cam.novell.com|164.99.129.35|:80… connected.
    HTTP request sent, awaiting response… 403 Forbidden
    2012-01-18 10:07:51 ERROR 403: Forbidden.

    Have any one faced a similar issue and if yes can you let me know how to come out from this

  15. Hi Reddy,
    You must have got your answer by now, posting info hopeing it would help others

    Connecting to cambuild.qalab.cam.novell.com|164.99.129.35|:80… connected.
    HTTP request sent, awaiting response… 403 Forbidden
    2012-01-18 10:07:51 ERROR 403: Forbidden.

    If you see the error above, it show your machine succesfully connected to remote host on port 80, however could not find the page you have requsted for403 Forbidden.
    Hope this clarify.

    1. This doesnt mean the page could not be found that’s a 404 error. This mean the user who attempted to login does not have the permission to access any further. Therefore “Forbidden” request error is thrown.

  17. I want to download two of three files (file-abc.zip, file-xyz.zip, sample.zip) from a ftp server. How can I use wget like————- file*.zip in a shell script.

  21. To protect yourself from having your password in your terminal history, you can use the following:

    read -s q; wget http://example.com/path/to/file --user=username --password="$q";

  22. واقعا ممنون به دردم خورد

    thanks a lot very Nice :x

  25. I want to execute this command using an automated process. Is there any way to hide or encrypt password?

  26. Hi Vivek,

    I’m completely new to Linux, and I’m setting up a development server at home using Ubuntu Server. Hours of, erm … fun? Anyway I found this really helpful.

    Many thanks

    Martin

    Have a question? Post it on our forum!

Tagged as: Tags , , , , , , ,