大网数据广告活动
网站运营与故障问答

Googlebot says access denied on non-existant resource

· 52CE 我爱测

Google Search Console is telling me: Access Denied Googlebot couldn't crawl your URL because your server either requires authentication to access the page, or it is blocking Googlebot from accessing your site. The URL looks somet…

问题原文

原帖链接 · Timothy B. · CC BY-SA 4.0

Google Search Console is telling me:

Access Denied

Googlebot couldn't crawl your URL because your server either requires authentication to access the page, or it is blocking Googlebot from accessing your site.

The URL looks something like this:

https://www.example.com/.html

I checked the URL in question and my server does indeed reply with a 403 Forbidden response and page.

This is not a URL in our sitemap.xml and we have never used it for anything. As far as I know, it has never been valid.

I tried the "Mark as Fixed" button in the Google Search Console, but the problem returned.

Any ideas how to properly handle without any detrimental effect to "link juice?" Thanks!

EDIT:

I believe I found the problem. This line in our Apache conf file:

  RewriteRule htm$ /$1.html [R=301,L]

There are no regex parentheses, so $1 will always be empty. So I believe the problem is caused by old backlinks hitting deprecated .htm URLs.

I replaced it with this:

  RewriteRule (.*)\.htm$ /$1.html [R=301,L]

已采纳解答

原帖链接 · DocRoot · CC BY-SA 3.0

Your server is probably configured to reject (with "403 Forbidden") all requests for dot files, ie. all files starting with a dot. These are usually config files, like .htaccess and .htpasswd, which should not be accessible publicly under any circumstance. These are naturally "hidden" on Linux, but not on Windows OS.

If you have access to the server config, then you can configure this, but otherwise this is normal behavior. Sometimes the server is configured to only block .ht* files, but otherwise a blanket "all dot files" should be blocked.

I tried the "Mark as Fixed" button in the Google Search Console, but the problem returned.

Unless you've changed the response on your server then this is not "fixed" and should not be marked as such in GSC.

This is not a "problem".

摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。

资料来源与适用范围

资料来源
Webmasters Stack Exchange
适用范围
以原帖系统、软件版本与发布日期为准(2018-04-26)
内容语言
中文;命令、路径和错误标识保留技术原文
本地收录日期
2026-11-15
授权说明
CC BY-SA 4.0 / CC BY-SA 3.0

正文与所需资源由本站本地存储。查看来源及更新记录 ↗

请按适用版本和实际日志确认原因;本站没有对每一种系统或设备逐项执行文中的操作。