How to Force Apache To Show a HTTP-410 Gone Status Code

in Categories , , , , last updated December 14, 2015

I removed www.example.com/foo/file.html from my domain. But, it is still being crawled by Google/Yahoo and many other bots. How do I return a HTTP-410 gone error i.e. tell these bots that resource permanently does not exist using Apache HTTPD server on a CentOS/RHEL/Fedora/Ubuntu/Debian and Unix-like operating system?

The http error status code 410 indicates that the resource (images,css,js and all other files) requested is no longer available and will not be available again ever. This 410 status code may be used when a resource has been intentionally deleted or part of unique url generated using some sort of token for given session (for example cached pdf file). Upon receiving a 410 status code, the client or bot should not request the resource again in the future. Bots such as search engines should remove the resource from their index. In this quick tutorial you will learn how to serve HTTP status code 410 from Apache rather than a 404 or 403 error code.

Configuration

The simplest configuration is to add the following line in .htaccess file using mod_alias Redirect directive. Cd to your DocumentRoot such as /var/www/html/:
$ cd /var/www/html/
$ vi .htaccess

Append the following line:
Redirect gone /foo/file.html
OR
Redirect 410 /foo/bar/demo.php
You can also use of regular expressions using RedirectMatch directive as follows:

# Syntax 
RedirectMatch gone regex-here
 
# Match all .png files in /foo/
RedirectMatch gone "/foo/\.png$"
 
# Another example for gif files starting with bar name
RedirectMatch gone "/foo/bar*\.png$"
 
# One more example. We now have resposive site so remove all old mobile friendly html pages
RedirectMatch gone "/mobilesite/*.html$"

Adding friendly message page

Although the Apache Server disparages generic error responses in the event of 4xx or 5xx HTTP status codes, these responses are rather stark, uninformative, and can be intimidating to site users. You may wish to provide custom error responses which are either friendlier, or in some language other than English, or perhaps which are styled more in line with your site layout. So append the following code:

ErrorDocument 410 /errorpages/410-mobile.gone.html

Save and close the file. Next create 410-mobile.gone.html in your DocumentRoot directory (for e.g. /var/www/html/errorpage/)
$ mkdir /var/www/html/errorpage/ && cd $_
$ vi 410-mobile.gone.html

Append the error message as per your needs:

<html>
<head>
   <title>Page Gone - 410 Error</title>
</head>
<body>
<blockquote>
<h1>Error 410 - Page deleted or gone</h1>
This might be because:
<ul>
  <li>You have typed the web address incorrectly, or the page you were looking for may have deleted.</li>
</ul>
Please try the following options instead:
<ul>
  <li>Use <a href="/search.html">search option</a> to see if it's available elsewhere. Or visit our home page for the latest info.</li>
</ul>
<hr>
<small>If you feel like it, mail the url, and where you came from to webmaster@example.com</small>
</blockquote>
</body>
</html>

Your visitor will see the page as follows along with HTTP/1.1 410 gone status code:

Fig.01: HTTP/1.1 410 gone HTML page sample
Fig.01: HTTP/1.1 410 gone HTML page sample

Verify error code

Simply type the following curl command:
$ curl -I www.example.com/foo/page.html
$ curl -I www.example.com/mobilesite/4242.html

Sample outputs:

HTTP/1.1 410 Gone
Server: Apache
Date: Mon, 14 Dec 2015 14:52:28 GMT
Content-Type: text/html
Content-Length: 335
Connection: keep-alive

Please note that you must get ‘HTTP/1.1 410 Gone’ as status code.

How do I generate a HTTP-410 error for complete domain?

Add the following to your VirtualHost or .htaccess. This is done using mod_rewrite:

   RewriteEngine On
   RewriteCond %{HTTP_HOST} ^www\.example\.com$ [NC]
   RewriteRule ^(.*)$ - [L,G]

Save and close the file.

References

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin, DevOps engineer, and a trainer for the Linux operating system/Unix shell scripting. Get the latest tutorials on SysAdmin, Linux/Unix and open source topics via RSS/XML feed or weekly email newsletter.

Share this on (or read 1 comments/add one below):

1 comment

  1. You could also use “RewriteCond ${REQUEST_FILENAME} !-f”, “RewriteCond ${REQUEST_FILENAME}.gone -f”, “RewriteRule ^(.*)$ – [L,G]” to return gone if the target file doesn’t exist, and the sentinal file does.

    Have a question? Post it on our forum!