问题原文
原帖链接 · Maximillian Laumeister · CC BY-SA 4.0
I have a custom 404.html file on my website that gets served with a 404 status code when a URL is not found, standard stuff. But if you visit the exact path /404.html on the server, the server says "hey, there's actually a file there" and shows you the 404 page but without the 404 status code.
I wanted to make sure that the /404.html path doesn't get accidentally indexed in search, so I added a <meta name="robots" content="noindex"> tag to it.
Of course, that means that my nonexistant URLs also have the meta noindex tag as part of the 404 page. Is there any way that the presence of this meta noindex tag could cause an issue for search engines / crawlers, or is this setup totally fine?
已采纳解答
原帖链接 · MrWhite · CC BY-SA 4.0
There's no problem having a meta noindex tag on the 404 page, to prevent 200 OK responses being indexed.
If this was a PHP page then you could obviously just send a 404 Not Found header as part of the standard response - to make sure that it always returns a 404.
If you are on the Apache web server then you could also use .htaccess (mod_rewrite) to force any direct requests to 404.html to also trigger a 404 and still serve your custom error document. For example:
RewriteEngine On
RewriteCond %{ENV:REDIRECT_STATUS} ^$
RewriteRule ^404\.html$ - [R=404]
The RewriteCond directive that checks the REDIRECT_STATUS environment variable is required if you want your custom 404 to be served (so that it only applies to direct requests). Otherwise, there will likely be a rewrite loop (500 error) when trying to serve the custom error document and you'll see the standard server generated 404 response (but it's still a 404).
摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。
