3 minutes read

SearchMasters built a tool to crawl the variations that homepage URLs often have, and provide advice based on the results. It has since been taken down, but the advice is still of great use.

Potential variations of a domain include

  • https://example.com
  • https://example.com
  • http://example.com
  • http://www.example.com
  • Each of the above with filenames such as /index.html, /index.php etc

Whilst every URL returns the same content, Google sees them as duplicates of each other which impacts how the website is indexed. It is important to set the preferred URL version sitewide, and do it correctly.

A Google bot crawling each of the URL variations may come across these response codes:

  • 200 - A valid page has been found at the specified URL
  • 404 - No page was found at the specified URL
  • 301 - The specified URL does a search engine friendly redirect to another page
  • 302 - The specified URL does a non search engine friendly redirect to another page
  • other - It would be unusual to receive a response other than those listed above

Generally speaking, only one of these should return a 200 response, and this is usually www.example.com

If any of the index.htm, index.php, index.asp, default.asp etc are returning a 200 response, you have a duplicate content problem. All these pages should return a 404 or a 301 response.

If you have https enabled, then there should be a 301 redirect from all http pages to their https version.

How to Safely Redirect to the Preferred URL In Apache .htaccess - not Windows, Cold Fusion etc

It is normal practice to have domains linked and accessed via the www version. However, if you prefer the non www version, see the alternate code shown below. If it does not work, please contact your host. There are many linux distributions, apache versions, php setups etc that can cause issues.

In a number of content management system (CMS) packages, the redirect should happen automatically. In others, .htaccess files should be checked, or CMS options or plugins investigated.

If HTTP_HOST does not work - try HTTP_X_FORWARDED_HOST.

To redirect all traffic from http://searchmasters.co.nz to https://www.searchmasters.co.nz (and also redirect all other domains parked into the same webspace)

RewriteEngine on RewriteCond %{HTTP_HOST} !^www\.searchmasters\.co\.nz RewriteRule (.*) https://www.searchmasters.co.nz/$1 [R=301,L]

RewriteCond %{HTTPS} off RewriteRule (.*) https://www.searchmasters.co.nz/$1 [R=301,L]

or to do the reverse: To redirect all traffic from http://www.searchmasters.co.nz to https://searchmasters.co.nz (and also redirect all other domains parked into the same webspace)

RewriteEngine on RewriteCond %{HTTP_HOST} !^searchmasters\.co\.nz RewriteRule (.*) https://searchmasters.co.nz/$1 [R=301,L]

RewriteCond %{HTTPS} off RewriteRule (.*) https://searchmasters.co.nz/$1 [R=301,L]

How to do a 301 redirect in PHP

If you ever need to do a redirect in PHP code, make sure that you do more than just the default - ie not just the R, but the R=301 in the htaccess, or not only the “Location”, but also the initial 301 Moved Permanently code. The default redirect is a non search engine friendly 302 temporary redirect. If you need to write a redirect in a different programing language, use the logic from the htaccess and php code to write you own redirects.

A 301 redirect is a search engine safe way redirecting traffic.

This PHP code must appear before any output is sent to the browser.

if(preg_match("/[index.php]$/",$_SERVER[REQUEST_URI])) { $url="http://".$_SERVER[HTTP_HOST].preg_replace("/(index.php)$/","",$_SERVER[REQUEST_URI]); $sapi=php_sapi_name(); if (PHP_VERSION >= '4.3.0') { header("Location: $url", 0, 301); } else if ($sapi == 'cgi' OR $sapi == 'cgi-fcgi') { header("Location: $url"); // Call the status header after Location so we are sure to wipe out the 302 header sent by PHP header('Status: 301 Moved Permanently'); } else { header("Location: $url"); header('HTTP/1.1 301 Moved Permanently'); } }