Speed up Apache 2.0 web access or downloads with mod_deflate

by on May 24, 2006 · 34 comments· LAST UPDATED September 13, 2007

in , ,

You can speed up downloads or web page access time with Apache mod_deflate module. The mod_deflate module provides the DEFLATE output filter that allows output from your server to be compressed before being sent to the client over the network.

This decreases the amount of time and data transmitted over the network, resulting in faster web experience or downloads for visitors.

Make sure mod_deflate included with your Apache server (by default it is now installed with all modern distro).

How can I speed up downloads from my Apache 2.0 server?

Open httpd.conf file using a text editor such as vi:
# vi httpd.conf

Append following line:
LoadModule deflate_module modules/mod_deflate.so

Append following configuration <Location /> directive:
<Location />
AddOutputFilterByType DEFLATE text/html text/plain text/xml
....
...
<Location>

Above line only compress html and xml files. Here is the configuration from one of my production box:
<Location />
...
...
AddOutputFilterByType DEFLATE text/plain
AddOutputFilterByType DEFLATE text/xml
AddOutputFilterByType DEFLATE application/xhtml+xml
AddOutputFilterByType DEFLATE text/css
AddOutputFilterByType DEFLATE application/xml
AddOutputFilterByType DEFLATE image/svg+xml
AddOutputFilterByType DEFLATE application/rss+xml
AddOutputFilterByType DEFLATE application/atom_xml
AddOutputFilterByType DEFLATE application/x-javascript
AddOutputFilterByType DEFLATE application/x-httpd-php
AddOutputFilterByType DEFLATE application/x-httpd-fastphp
AddOutputFilterByType DEFLATE application/x-httpd-eruby
AddOutputFilterByType DEFLATE text/html
...
...
<Location>

Close and save the file. Next restart apache web server. All of the above extension file should compressed by mod_deflate:
# /etc/init.d/httpd restart

You can also specify specific directory and enabling compression only for the html files. For example /static/help/ directory:
<Directory "/static/help">
AddOutputFilterByType DEFLATE text/html
</Directory>

In real life, there are issues with compressing other types of files such as mp3 or images. If you don't want to compress images or mp3 files, add following to your configuration:
SetOutputFilter DEFLATE
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.(?:exe|t?gz|zip|bz2|sit|rar)$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.pdf$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.avi$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.mov$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.mp3$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.mp4$ no-gzip dont-vary
SetEnvIfNoCase Request_URI \.rm$ no-gzip dont-vary

Please note that this processing takes additional CPU and memory on your server as well as on the client browser. So you must make decision which document you need to compress (thanks to mdxp).

See also:

TwitterFacebookGoogle+PDF versionFound an error/typo on this page? Help us!

{ 34 comments… read them below or add one }

1 John May 25, 2006 at 8:21 pm

And how the hell I am gonna test a page?

Use lwp-request command, which is a simple command line user agent.

lwp-request -e -H ‘Accept-Encoding: gzip’ http://www.yoursite.com/

Cheers,

John

Reply

2 nixCraft May 25, 2006 at 8:25 pm

John,

Yes, you can test it with lwp-request.

Appreciate your post.

Reply

3 l33t May 26, 2006 at 8:58 pm

Good article but please note that:

a) mod_deflate has replaced Apache 1.3′s mod_gzip in Apache2 i.e. mod_gzip is not at all compatible with Apache 2.0. I had big problem with mod_gzip when upgraded to Apache 2.0 (FreeBSD box)

b) mod_deflate should work with all modern browsers but it will not work with outdated browser such as Netscape v4.0, just add following rules:
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html

See the mod_deflate home page for more examples

Reply

4 tony May 26, 2006 at 9:06 pm

I’m using Apache 2.0.x with mod_deflate and I have no trouble.

I carried out some experiments with both mod_gzip and mod_deflate. My results were as follows:

File: test.php (size w/o compression: 53664 bytes)
Apache 2.0 with mod_deflate: 17709
Apache 1.3.x with mod_gzip: 22002


Tony

Reply

5 mangoo May 29, 2006 at 1:15 pm

Is the configuration similar for Apache 2.2.x?

Reply

6 nixCraft May 29, 2006 at 1:19 pm

>Is the configuration similar for Apache 2.2.x?

Yes

Reply

7 mdxp May 29, 2006 at 1:29 pm

You might want to add somewhere, that this will obviously add some additional CPU overhead, because all the operation (to compress the files before sending it to the client browser) is done in real time. This might not cause problems with small sites, but on higher traffic sites it can have an impact on the performance of the system.

Reply

8 MikeTA May 29, 2006 at 1:45 pm

I can really vouch for mod_deflate, it is particularly good on sites where pages are generated dynamically (ie, from CGI scripts) and contain a lot of repeated boilerplate.

I’d also recommend a look at mod_expires if your pages contain a lot of stock graphics (such as icon-like images) that appear on lots of pages and which never (or infrequently) change.

Reply

9 nixCraft May 29, 2006 at 3:18 pm

MikeTA, good hint, mod_expires seems to be a good choice for all those tiny graphics files. Earlier I had used 512Mb/1GiB ramdisk mounted on /images, to server images from ram to save disk I/O. I will try out mod_expires …

Mdxp, considering power of today CPU and lots RAM it will not make big difference. But, I will keep an eye on this issue, thanks for heads up.

Appreciate your post.

Reply

10 Razvan May 29, 2006 at 5:24 pm

How is this technique working with the major browsers ?

Internet Explorer
Firefox
Mozilla
Opera

? ? ?

What is happening if a page is still very slow to load ? If the page is just HTML (uncompressed) the browser will start to render it as it loads. At least the used doesn’t have to wait until the whole page is downloaded.

If this module is used the whole compressed page must be downloaded before anything is displayed ? True / False

If the above is true, then webmasters should check that the sizes of the compressed files are not bigger than 50k.

Are there any popular web sites (more than 100,000 unique visitors a month) that are using this technique ? Can you provide some examples ?

Regards,
Razvan

Reply

11 nixCraft May 29, 2006 at 6:26 pm

Razvan,

It works with all modern browsers but it may not work with old browser such as Netscape or IE. You can setup filter to ignore unsupported browser.

Please note that if a client web browser requests and tells server that it supports compression then server sent back compressed document (otherwise server will send normal uncompressed document). So you don’t have to worry about browsers.

It is *quite* useful if end-user is connected via modem.

Slashdot and many other sites with tons of visitors use thing technique You can see if particular site is using mod_gzip or mod_deflate http://www.whatsmyip.org/mod_gzip_test/

And remember this processing takes additional CPU and memory on your server as well as on the client browser. So you must make decision which document you need to compress.

Reply

12 Scott May 29, 2006 at 9:17 pm

Also, keep in mind that mod_deflate must queue all transfers in memory in order to compress the content prior to delivery to the client. Keep an eye on your server’s memory utilization if you enable this on a busy site.

All in all, it works very well on the sites I manage which average 12 million hits/month.

Scott

Reply

13 Hans-Werner July 26, 2006 at 11:10 am

@Razvan and Scott: gzip is a stream compressor, it doesn’t need random access to the full data to be (un-)compressed. Such no full buffering is needed, neither server side, nor client side.

Reply

14 GoLD October 21, 2006 at 12:14 am

Wich value you use for the DeflateCompressionLevel directive ?
Thanks for all. :)

Reply

15 Farrukh Shahzad September 13, 2007 at 9:18 am

“Hans-Werner”,
what you are saying I don’t agree to that…

To use the gzip compression in PHP/Apache the output buffering is required to be set as “ON”

So whole page (or output buffer gets full) is buffered and compressed and then sent to the client….

also the funny part is that if a client receives a compress page and that page contains more than one gzip compress block then till all the blocks are downloaded at client browser… the output in the browsers will not be displayed…

mean page will show up till all the compressed contents are downloaded in the client side

Reply

16 Nigel Aves November 28, 2007 at 2:11 am

Thanks for this incredibly useful page, integrated into Apache with no problems.

One technical question. Does anyone know why “image” files can not be compressed? I run an on-line gallery so even a few % of compression could be handy.

The final should be

Log Files …

Found this on the apache site

DeflateFilterNote Input instream
DeflateFilterNote Output outstream
DeflateFilterNote Ratio ratio

LogFormat ‘”%r” %{outstream}n/%{instream}n (%{ratio}n%%)’ deflate
CustomLog logs/deflate_log deflate

solves the question above about “testing”. Seems to work with webalizer as well.

Many Thanks

Nigel.

Reply

17 Nigel Aves November 28, 2007 at 2:13 am

ah …. this was dropped from my message :)

the final location should be preceded by a back-slash.

Reply

18 Jeff Shields December 11, 2007 at 4:08 am

I used Mod_deflate with apache 2.0.54 with .shtml SSI which also have embedded php code. These pages are returned empty to the browser.

Not sure why this is happening. Anyone know about this.

Reply

19 matsu March 23, 2008 at 5:10 pm

I found spelling mistake.

The end of code snipet, should be .

thank you for your article. Very useful for real operating server.

Reply

20 Shivin August 14, 2008 at 9:04 am

Hi,

Will mod_jk and mod_deflate works together?
If yes which versions are good.
Pls mail me the answer shivinvijai@gmail.com

Thanks in advance
Shivin

Reply

21 treblam November 8, 2008 at 6:18 am

The end tag should be ,if I use I would get this error when I restart apache:”The requested operation has failed”

Reply

22 Jiwei Wang November 28, 2008 at 3:20 am

Great tip. Works like a charm, speeding up my sites a good percentage. One bug in the post: should read in two places.

Reply

23 Jiwei Wang November 28, 2008 at 3:23 am

My previous comment was messed up by the html code. The bug is “Location” should read “/Location”, replacing ” with angle brackets.

Reply

24 Anton Ongsono December 25, 2008 at 11:02 am

how to use this in htaccess?

Reply

25 otyek james January 29, 2009 at 10:06 am

thanxs alot, we in africa have issues woth bandwidth, I believe this will help optimize our webserver’s performance.
Good day,
Ronald

Reply

26 Amol February 9, 2009 at 5:57 am

Can we ask apache not to compress the static contents every time and use the compressed version which is would have prepared for the first time access

Reply

27 Amol February 9, 2009 at 6:03 am

We have enabled apache compression(mod_deflate) as per the configuration given
—-config—–
LoadModule deflate_module modules/mod_deflate.so

# Insert filter
SetOutputFilter DEFLATE

# Netscape 4.x has some problems…
BrowserMatch ^Mozilla/4 gzip-only-text/html

# Netscape 4.06-4.08 have some more problems
BrowserMatch ^Mozilla/4\.0[678] no-gzip

# MSIE masquerades as Netscape, but it is fine
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
## Don’t compress images
SetEnvIfNoCase Request_URI \
\.(?:gif|jpe?g|png)$ no-gzip dont-vary

# Make sure proxies don’t deliver the wrong content
Header append Vary User-Agent env=!dont-vary

—————–

And after this we are facing one strange issue, that if we access the contents from the local intranet, we get the compressed version. However if we access the same contents over the internet(using the same url, our site is static one), we get the un-compressed version, we have identified this by observing the bytes delivered for the request. We also tried enabling/disabling the proxy setting, but no change. Any reason/clue for such kinda situation?

thanks

Reply

28 David December 16, 2009 at 6:08 am

How do I set mod_deflate to work for just ONE domain, not all the domains that my Apache web server works for?

Reply

29 Daniel January 27, 2010 at 10:53 am

@David
Just add the configurationdata to the vhost container for your particular domain.

Reply

30 sim so dep January 27, 2011 at 12:21 pm

Hey cool weblog, simply questioning what anti-spam software program you utilize for comments because i get heaps on my blog. Anyway, in my language, there are not much good source like this.

Reply

31 Fuming Solder May 18, 2011 at 12:51 am

Somehow I still have issues with IE8 corrupting zip files even though zip files are specifically excluded from further gzipping with

SetEnvIfNoCase Request_URI \.(?:exe|t?gz|zip|bz2|sit|rar)$ no-gzip dont-vary

Does anyone have a clue why this regex mey not be working with Apache/2.2.11 (Unix) ? Or is there another reason Apache keeps gzipping zip files (and IE8 does not recognize that the file has been compressed twice) ?

Thanks!

Reply

32 Triactol August 11, 2011 at 7:54 am

I am not sure where you’re getting your information, but good topic. I needs to spend some time learning much more or understanding more. Thanks for excellent info I was looking for this info for my mission.

Reply

33 Eloy Leja September 19, 2011 at 4:15 am

Hi there can I quote some of the material here in this site if I reference you with a link back to your site?

Reply

34 Vidyadhar January 14, 2012 at 8:28 am

0 down vote favorite
share [fb] share [tw]

I want to enable compression in apache. For the same I am trying to use deflate module of apache.

I have already installed apache 2. But it is a default installation and in that mod_deflate is not enable. Now I am compiling mod_deflate module by using following commands:

cd /usr/local/src/httpd-2.2.21/modules/filters/
/usr/local/apache/bin/apxs -i -c -Wl,lz mod_deflate.c

After doing it I have added following line in my httpd.conf file

LoadModule deflate_module modules/mod_deflate.so

but as soon as i do syntax check I am getting following error httpd: Syntax error on line 417 of /usr/local/apache/conf/httpd.conf: Cannot load /usr/local/apache/modules/mod_deflate.so into server: /usr/local/apache/modules/mod_deflate.so: undefined symbol: inflate

Following is the output of ldd command

linux-gate.so.1 => (0x00b2d000)
libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0×00110000)
/lib/ld-linux.so.2 (0x008e6000)

Please help as i dont want to compile apache again.

Reply

Leave a Comment

Tagged as: , , , , , , ,

Previous post:

Next post: