问题原文
原帖链接 · alexus · CC BY-SA 3.0
My code base is shared between several environments (live, staging, dev) & sub-domains (staging.example, dev.example, etc.) and only two should be allowed to be crawled (ie. www.example and example). Normally I'd modify /robots.txt and add Disallow: /, but due to shared code base I cannot modify /robots.txt without affecting all (sub)domains.
Any ideas how to go about it?
已采纳解答
原帖链接 · MrWhite · CC BY-SA 3.0
You can serve a different robots.txt file based on the subdomain through which the site has been accessed. One way of doing this on Apache is by internally rewriting the URL using mod_rewrite in .htaccess. Something like:
RewriteEngine On
RewriteCond %{HTTP_HOST} !^(www\.)?example\.com$ [NC]
RewriteRule ^robots\.txt$ robots-disallow.txt [L]
The above states that for all requests to robots.txt where the host is anything other than www.example.com or example.com, then internally rewrite the request to robots-disallow.txt. And robots-disallow.txt will then contain the Disallow: / directive.
If you have other directives in your .htaccess file then this directive will need to be nearer the top, before any routing directives.
摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。
