≡ Menu

wget Command: Download Compressed File By Sending gzip Headers

You can send HEADER-LINE along with the rest of the headers in each HTTP request. The supplied header is sent as-is, which means it must contain name and value separated by colon, and must not contain newlines. You may define more than one additional header by specifying --header more than once as follows:

wget --header='Accept-Charset: iso-8859-2' --header='Accept-Language: hr'  http://server1.cyberciti.biz/file.css

Example: Testing gzip encoding with wget command

To send gzip encoding request, enter:
$ wget --header='Accept-Encoding: gzip' http://www.cyberciti.biz/hardware/linux-find-and-recover-wasted-disk-space/
Sample outputs:

--2012-10-28 17:48:06--  http://www.cyberciti.biz/hardware/linux-find-and-recover-wasted-disk-space/
Resolving www.cyberciti.biz... 75.126.153.206
Connecting to www.cyberciti.biz|75.126.153.206|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: `index.html.54'
 
    [ <=>                                  ] 12,657      --.-K/s   in 0.02s   
 
2012-10-28 17:48:07 (583 KB/s) - `index.html.54' saved [12657]

Download the sample page without gzip:
$ wget http://www.cyberciti.biz/hardware/linux-find-and-recover-wasted-disk-space/
Sample outputs:

--2012-10-28 17:48:37--  http://www.cyberciti.biz/hardware/linux-find-and-recover-wasted-disk-space/
Resolving www.cyberciti.biz... 75.126.153.206
Connecting to www.cyberciti.biz|75.126.153.206|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: `index.html.55'
 
    [   <=>                                ] 45,729      73.7K/s   in 0.6s    
 
2012-10-28 17:48:38 (73.7 KB/s) - `index.html.55' saved [45729]

From the above two outputs:

  1. gzip enabled page was downloaded in 0.2 seconds using wget command.
  2. Without gzip page was downloaded in 0.6 seconds using wget command.

Use this option to test:

  1. Testing and troubleshooting HTTP server problems
  2. CDN edge node speed.
  3. Your origin server speed.
  4. Web server gzip comparability.
  5. Load balancer / reverse proxy server testing.

As of wget v1.10, this option can be used to override headers otherwise generated automatically. In this example wget is used connect to www.cyberciti.biz, but to specify ‘beta.cyberciti.biz’ in the Host header (i.e. show page from beta.cyberciti.biz for same domain name :

wget --header="Host: beta.cyberciti.biz" http://www.cyberciti.biz/

Finally, you can ave the headers sent by the HTTP server to the file, run:
$ wget --save-headers http://www.cyberciti.biz
$ vi index.html

Sample outputs:

Unix / Linux: Wget Save The Http Headers Command

Fig.01: wget saving the http headers

Share this tutorial on:
{ 0 comments… add one }
Security: Are you a robot or human?

Leave a Comment

You can use these HTML tags and attributes: <strong> <em> <pre> <code> <a href="" title="">


   Tagged with: , , , ,