Download a file with curl on Linux / Unix command line

last updated in Categories , , ,

I am a new macOS Unix user. I am writing a small bash shell script. How do I download files straight from the command-line interface using curl? How can I download files with cURL on a Linux or Unix-like systems?

Introduction : cURL is both a command line utility and library. One can use it to download or transfer of data/files using many different protocols such as HTTP, HTTPS, FTP, SFTP and more.

The curl command line utility lets you fetch a given URL or file from the bash shell. This page explains how to download files with curl command on a Linux, macOS, *BSD and Unix-like operating systems.

How to download a file with curl command

The basic syntax is:
curl url --output filename
curl https://url -o output.file.name

Let us try to download a file from https://www.cyberciti.biz/files/sticker/sticker_book.pdf and save it as output.pdf
curl https://www.cyberciti.biz/files/sticker/sticker_book.pdf -o output.pdf
OR
curl https://www.cyberciti.biz/files/sticker/sticker_book.pdf --output output.pdf
Download a file with curl command
The -o or --output option allows you to give the downloaded file a different name. If you do not provide the output file name curl will display it to the screen.

How to download a single file without giving output name

You can save output file as it is i.e. write output to a local file named like the remote file we get. For example, sticker_book.pdf is a file name for remote URL https://www.cyberciti.biz/files/sticker/sticker_book.pdf. One can save it sticker_book.pdf directly without specifying the -o or --output option by passing the -O (capital
curl -O https://www.cyberciti.biz/files/sticker/sticker_book.pdf

Downloading files with curl
Downloading files with curl in a single shot

How to deal with HTTP 301 redirected file

The remote HTTP server might send a different location status code when downloading files. For example, HTTP URLs are often redirected to HTTPS URLs with HTTP/301 status code. Just pass the -L follow the 301 (3xx) redirects and get the final file on your system:
curl -L -O http://www.cyberciti.biz/files/sticker/sticker_book.pdf

How to download multiple files using curl

Try:
curl -O url1 -O url2
curl -O https://www.cyberciti.biz/files/adduser.txt \
     -O https://www.cyberciti.biz/files/test-lwp.pl.txt

One can use the bash for loop too:

## define a bash shell variable ##
urls="https://www.cyberciti.biz/files/adduser.txt https://www.cyberciti.biz/files/test-lwp.pl.txt"
 
## let us grab it ##
for u in $urls
do
   curl -O "$u"
done
How to download a file using curl and bash for loop
How to download a file using curl and bash for loop

How to download a password protected file with curl

Try any one of the following syntax
curl ftp://username:passwd@ftp1.cyberciti.biz:21/path/to/backup.tar.gz
curl --ftp-ssl -u UserName:PassWord ftp://ftp1.cyberciti.biz:21/backups/07/07/2012/mysql.blog.sql.tar.gz
curl https://username:passwd@server1.cyberciti.biz/file/path/data.tar.gz
curl -u Username:Password https://server1.cyberciti.biz/file/path/data.tar.gz

How to download file using a proxy server

Again syntax is as follows:
curl -x proxy-server-ip:PORT -O url
curl -x 'http://vivek:YourPasswordHere@10.12.249.194:3128' -v -O https://dl.cyberciti.biz/pdfdownloads/b8bf71be9da19d3feeee27a0a6960cb3/569b7f08/cms/631.pdf

How to use curl command with proxy username/password
How to use curl command with proxy username/password

Conclusion

Like most Linux or Unix CLI utilities, you can learn much more about curl command by visiting this help page.

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin, DevOps engineer, and a trainer for the Linux operating system/Unix shell scripting. Get the latest tutorials on SysAdmin, Linux/Unix and open source topics via RSS/XML feed or weekly email newsletter.

Start the discussion at www.nixcraft.com

Historical Comment Archive

3 comment

  1. I always set output file with
    curl -o linux.tar.gz https://cdn.kernel.org/pub/linux/kernel/v3.x/linux-3.0.13.tar.gz

  2. This is nice. Here is one more hint, if you use self signed SSL cert, I can get it verified it with the following:
    curl --cacert my-ssl.crt -O https://my-ip/my-file.tgz

    Cheers mate

  3. So here is the thing. I am download certain mp4 files and the remote http server limiting the connection. So after a bit of reading and thanks to your for loop example, I can get around by setting timeout time value. I found this in man page:

    Maximum time in seconds that you allow curl’s connection to take. This only limits the connection phase, so if curl connects within the given period it willontinue – if not it will exit

    So I did something:

    urls="url1 url2 ... url100"
    for u in $urls
    do 
      curl --connect-timeout=7 -O $u
    done
    

    I hope it might help someone.

    Have a question? Post it on our forum!