Less than 1 minute read

How do you have several sites coming into the one hosting space, and have one with different robots.txt file. Correct usage of the .htaccess file enables multiple robots files.

Logic - if accessing via domainb.com/robots.txt, then show the robotsfff.txt file, otherwise show the default robots.txt file.

.htaccess file includes:

RewriteCond %{HTTP_HOST} domainb.com
RewriteRule robots.txt robotsfff.txt [L]

Then the robotsfff.txt file containing say a total exclusion:

User-agent: * Disallow: //

this means that the domainb.com/robots.txt shows as User-agent: * Disallow: //

And no need to redirect the access for the robotsfff.txt file. You could do a 301 redirect back to the robots.txt file, not worth the effort.