≡ Menu

How to install and configure Varnish cache on Ubuntu Linux 16.04 LTS

How do I install Varnish Cache server to improve the performance of my existing Apache web server especially caching dynamic content on Ubuntu Linux 16.04 LTS server?

Varnish Cache is a web application accelerator. You install it in front of any web-server that speaks HTTP and configure it to cache the contents. It act as a web application accelerator focusing on optimizing caching and compression It is really fast and used by high traffic websites. Let us see how to configure Varnish cache version 4.x on Ubuntu Linux 16.04 LTS server.

Our setup

Fig.01: Set up Varnish cache with various options

Fig.01: Set up Varnish cache with various options

Install Varnish

Type the following apt-get command/apt command to install Varnish cache software:
$ sudo apt update
$ sudo apt upgrade
$ apt search Varnish
$ sudo apt install varnish

Sample outputs:

Reading state information... Done
The following additional packages will be installed:
  binutils cpp cpp-5 gcc gcc-5 libasan2 libatomic1 libcc1-0 libcilkrts5 libgcc-5-dev libgomp1 libisl15 libitm1 libjemalloc1 liblsan0 libmpc3 libmpx0 libquadmath0 libtsan0
  libubsan0 libvarnishapi1
Suggested packages:
  binutils-doc cpp-doc gcc-5-locales gcc-multilib make autoconf automake libtool flex bison gdb gcc-doc gcc-5-multilib gcc-5-doc libgcc1-dbg libgomp1-dbg libitm1-dbg
  libatomic1-dbg libasan2-dbg liblsan0-dbg libtsan0-dbg libubsan0-dbg libcilkrts5-dbg libmpx0-dbg libquadmath0-dbg varnish-doc
The following NEW packages will be installed:
  binutils cpp cpp-5 gcc gcc-5 libasan2 libatomic1 libcc1-0 libcilkrts5 libgcc-5-dev libgomp1 libisl15 libitm1 libjemalloc1 liblsan0 libmpc3 libmpx0 libquadmath0 libtsan0
  libubsan0 libvarnishapi1 varnish
0 upgraded, 22 newly installed, 0 to remove and 0 not upgraded.
Need to get 23.1 MB of archives.
After this operation, 79.7 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://in.archive.ubuntu.com/ubuntu xenial/main amd64 libmpc3 amd64 1.0.3-1 [39.7 kB]
.....
...
..
Setting up libquadmath0:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Setting up libgcc-5-dev:amd64 (5.4.0-6ubuntu1~16.04.4) ...
Setting up gcc-5 (5.4.0-6ubuntu1~16.04.4) ...
Setting up gcc (4:5.3.1-1ubuntu1) ...
Setting up libvarnishapi1 (4.1.1-1) ...
Setting up libjemalloc1 (3.6.0-9ubuntu1) ...
Setting up varnish (4.1.1-1) ...
Processing triggers for libc-bin (2.23-0ubuntu5) ...
Processing triggers for systemd (229-4ubuntu16) ...
Processing triggers for ureadahead (0.100.0-19) ...

Varnish installed and it is configured to listen on port 6081. This is default. You can verify it with the following simple netstat command:
$ netstat -tulpn | grep varnishd
Sample outputs:

Fig.02: Varnish is configured to listen on port TCP IPv4/IPv6 port # 6081

Fig.02: Varnish is configured to listen on port TCP IPv4/IPv6 port # 6081

Varnish configuration

Ubuntu Linux 16.04 LTS uses systemd based config files as follows:

  • /lib/systemd/system/varnish.service – Default config file
  • TCP 6081 port – Varnish cache port
  • TCP 6082 port – Varnish management port
  • /etc/varnish/ – Varnish configuration files including VCLs

Let us see how to configure as per setup #1 (see fig.01).

Step 1 – Put Varnish cache on port 80

Type the following command to edit varnish cache file and create a new config file /etc/systemd/system/varnish.service.d/:
$ sudo systemctl edit varnish.service
Add/append the following config options:

[Service]
ExecStart=
ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a 202.54.2.50:80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,1024m

Save and close the file. Where,

  1. 202.54.2.50:80 – Listen to public IP address on port 80.
  2. 1024m – Increase RAM usage for Varnish from default 256m to 1024m

Type the following command to reload changes:
$ sudo systemctl daemon-reload

Step 2 – Configure Varnish to use our Apache server as a backend

Next, you will configure Varnish to use our Apache server as a backend which is running on IP address 192.168.1.6 port 80. Edit the /etc/varnish/default.vcl, enter:
$ sudo vi /etc/varnish/default.vcl
Set the following to point to your content server (Apache/Lighttpd/PHP/Python app server) running on 192.168.1.6 port 80.

backend default {
    .host = "192.168.1.6";
    .port = "80";
}

Basically, you need to set the values of host and port match your LAMP server private IP address and listening port. Save and close the file.

Step 3 – Configure Apache to use run on IP address 192.168.1.6 port 80

Type the following command to edit the /etc/apache2/ports.conf:
$ sudo vi /etc/apache2/ports.conf
Edit/add/append Listen directive:

Listen 192.168.1.6:80

Save and close the file. Edit /etc/apache2/sites-enabled/000-default.conf, enter:
$ vi /etc/apache2/sites-enabled/000-default.conf
The Virtual Host should be updated as follows :

 <VirtualHost 192.168.1.6:80>

Save and close the file. Finally, restart the Apache web-server, run:
$ sudo systemctl restart apache2.service
At this stage you need to restart the Varnish cache too, enter:
$ sudo systemctl restart varnish.service

Step 4 – Configure ufw to open port 80

Type the following command to open port 80 using ufw:
$ sudo ufw allow 80
Sample outputs:

Rule added
Rule added (v6)

Step 5- Test it

Fire your webbrowser and type the url or IP address:

http://your-ip-here/
http://202.54.2.50/
http://your-domain-name/

Sample outputs:

Fig.03: Varnish working on my setup

Fig.03: Varnish working on my setup


Another option is to use the curl command to see http headers:
$ url -I http://202.54.2.50/
Sample outputs:

HTTP/1.1 200 OK
Date: Wed, 22 Feb 2017 11:59:38 GMT
Server: Apache/2.4.18 (Ubuntu)
Last-Modified: Wed, 22 Feb 2017 11:42:57 GMT
Vary: Accept-Encoding
Content-Type: text/html
X-Varnish: 32773 32771
Age: 4
Via: 1.1 varnish-v4
ETag: W/"2c39-5491cff7c5c13-gzip"
Accept-Ranges: bytes
Connection: keep-alive

The value of Age: indicate that pages has been cached by Varnish server.

How do I see the Varnish log and other info?

To see Varnish log entry ranking, enter:
$ sudo varnishtop
Sample outputs:

Fig.04: varnishtop in action

Fig.04: varnishtop in action


To see log file cd to /var/log/varnish
$ cd /var/log/varnish
$ sudo tail -f varnish.log
$ sudo tail -f varnishncsa.log

Sample outputs (provided that you configured PHP/Python with Apache you should see something as follows):

151.1.xx.yy - - [22/Feb/2017:12:03:48 +0000] "GET http://www.cyberciti.biz/faq/ubuntu-linux-start-restart-stop-apache-web-server/ HTTP/1.0" 200 60944 "https://www.google.co.in/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:48.0) Gecko/20100101 Firefox/48.0"
111.z.xx.yy  - - [22/Feb/2017:12:03:48 +0000] "GET http://www.cyberciti.biz/faq/linux-disk-format/ HTTP/1.0" 200 94830 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
xxx.ttt.zz.tt - - [22/Feb/2017:12:03:48 +0000] "GET http://www.cyberciti.biz/faq/wp-content/plugins/contextual-related-posts/default.png HTTP/1.0" 200 15454 "https://www.cyberciti.biz/faq/ubuntu-linux-start-restart-stop-apache-web-server/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:48.0) Gecko/20100101 Firefox/48.0"
aa.bb.cc.dd  - - [22/Feb/2017:12:03:48 +0000] "GET http://www.cyberciti.biz/faq/find-linux-distribution-name-version-number/ HTTP/1.0" 200 92470 "https://www.google.co.in/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0"

See the Varnish server cache statistics

Type the following command:
$ sudo varnishstat
$ sudo varnishstat -1
$ sudo varnishstat -1 | grep -i cache

Sample outputs:

MAIN.cache_hit            334059         6.33 Cache hits
MAIN.cache_hitpass           239         0.00 Cache hits for pass
MAIN.cache_miss            94300         1.79 Cache misses

How do I purge content from the Varnish cache?

First you need to define ACL. I have the following config in my /etc/varnish/default.vcl file:

# Who is allowed to purge?
acl purge {
        "localhost";
        "127.0.0.1";
        "72.14.190.12";
        "192.168.1.6";
}
sub vcl_recv {
        # allow PURGE from localhost,72.14.190.12 and 192.168.1.6
        if (req.method == "PURGE") {
                if (!client.ip ~ purge) {
                        return(synth(405,"Not allowed."));
                }
                return (purge);
        }
}

Use the following command to invalidate https://www.cyberciti.biz/faq/bash-for-loop/ page I would call out to Varnish like this:
$ varnishadm -T 127.0.0.1:6082 -S /etc/varnish/secret ban "req.http.host == www.cyberciti.biz && req.url == /faq/bash-for-loop/"
To remove/purge all .html pages from cache, enter:
$ varnishadm -T 127.0.0.1:6082 -S /etc/varnish/secret "ban req.http.host ~ www.cyberciti.biz && req.url ~ .html"
If you want to purge everything just restart the Varnish cache:
$ sudo systemctl restart varnish.service
See HTTP Purging for more info. Stay tunned for more Nginx SSL config and creating custom VCL for caching pages.

See also

Share this tutorial on:

Your support makes a big difference:
I have a small favor to ask. More people are reading the nixCraft. Many of you block advertising which is your right, and advertising revenues are not sufficient to cover my operating costs. So you can see why I need to ask for your help. The nixCraft, takes a lot of my time and hard work to produce. If you use nixCraft, who likes it, helps me with donations:
Become a Supporter →    Make a contribution via Paypal/Bitcoin →   

Don't Miss Any Linux and Unix Tips

Get nixCraft in your inbox. It's free:



{ 0 comments… add one }

Leave a Comment

You can use these HTML tags and attributes: <strong> <em> <pre> <code> <a href="" title="">


   Tagged with: , , , , , ,