大网数据广告活动
服务器运维问答

AH01797: client denied by server configuration: /usr/share/doc

· 52CE 我爱测

Since quite a while (over a month now) I see lines like the following in the apache logs: 180.76.15.138 - - [24/Jun/2015:16:13:34 -0400] "GET /manual/de/mod/module-dict.html HTTP/1.1" 403 396 "-" "Mozilla/5.0 (compatible; Baidusp…

问题原文

原帖链接 · matpen · CC BY-SA 3.0

Since quite a while (over a month now) I see lines like the following in the apache logs:

180.76.15.138 - - [24/Jun/2015:16:13:34 -0400] "GET /manual/de/mod/module-dict.html HTTP/1.1" 403 396 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
180.76.15.159 - - [24/Jun/2015:16:28:34 -0400] "GET /manual/es/mod/mod_cache_disk.html HTTP/1.1" 403 399 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
66.249.75.86 - - [24/Jun/2015:16:18:01 -0400] "GET /manual/es/programs/apachectl.html HTTP/1.1" 403 436 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
[Wed Jun 24 16:13:34.430884 2015] [access_compat:error] [pid 5059] [client 180.76.15.138:58811] AH01797: client denied by server configuration: /usr/share/doc/apache2-doc/manual/de/mod/module-dict.html
[Wed Jun 24 16:18:01.037146 2015] [access_compat:error] [pid 2791] [client 66.249.75.86:56362] AH01797: client denied by server configuration: /usr/share/doc/apache2-doc/manual/es/programs/apachectl.html
[Wed Jun 24 16:28:34.461298 2015] [access_compat:error] [pid 2791] [client 180.76.15.159:25833] AH01797: client denied by server configuration: /usr/share/doc/apache2-doc/manual/es/mod/mod_cache_disk.html

The requests seem to really come from Baiduspider and Googlebot (checked using reverse DNS as explained here):

user@server:~$ host 66.249.75.86
86.75.249.66.in-addr.arpa domain name pointer crawl-66-249-75-86.googlebot.com.
user@server:~$ host crawl-66-249-75-86.googlebot.com
crawl-66-249-75-86.googlebot.com has address 66.249.75.86

I have read similar questions about this topic like this and this, but for those, these errors are actually preventing the site to work correctly. In my case instead, the html pages that the bots try to access do not exist, and this is therefore the expected behaviour of Apache. Only annoyance, is that Google seems slow at indexing my site, although the Google Webmaster Tools do not show any errors.

I am using Apache version 2.4.7 with the following vhost configuration:

<VirtualHost *:80>
    ServerName example.com
    ServerAlias www.example.com

    DocumentRoot "/var/www/example.com/public"
    <Directory />
        Options None
        AllowOverride None
        Order Deny,Allow
        Deny from all
        Require all denied
    </Directory>
    <Directory "/var/www/example.com/public">
        Options None
        AllowOverride FileInfo Limit Options=FollowSymLinks 
        Order Allow,Deny
        Allow from all
        Require all granted
    </Directory>

    ErrorLog /var/log/apache2/example.com/error.log
    CustomLog /var/log/apache2/example.com/access.log combined
</VirtualHost>

My questions are therefore:

  1. why are Baiduspider and Googlebot repeatedly trying to access content on my site which is not there and not referred by any links on the site?
  2. how do requests like GET /manual/de/mod/... get mapped to /usr/share/doc/apache2-doc/manual/de/mod/... while, to my understanding, they should go to /var/www/example.com/public/manual/de/mod/...?
  3. in general: should I worry about those lines as a sign of misconfiguration, or is there an explanation for them?

已采纳解答

原帖链接 · matpen · CC BY-SA 3.0

Since some time has passed without any answer, I decided to (partially) answer my own question according to my research so far.

  1. Unfortunately, the question why Googlebot and Baiduspider are trying to access the Apache documentation through my server remains unanswered.
  2. The /manual/... URLs get mapped to /usr/share/doc/apache2-doc/manual/... thanks to a pre-installed Alias on Ubuntu: I guess that this is so, in order to make it convenient to access the documentation. In my case this is not needed, so I decided to remove the Alias by issuing a2disconf apache2-doc followed by service apache2 reload.
  3. There is no reason to regard the log entries as signs of misconfiguration, as they are rather the desired behaviour. Before removing the Alias, the access to the documentation was blocked by the vhost config, thus returning a 403 "Forbidden" status code. After removing the Alias, the server correctly returns a 404 "Not Found" status code.

摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。

资料来源与适用范围

资料来源
Server Fault
适用范围
以原帖系统、软件版本与发布日期为准(2015-06-25)
内容语言
中文;命令、路径和错误标识保留技术原文
本地收录日期
2026-11-15
授权说明
CC BY-SA 3.0

正文与所需资源由本站本地存储。查看来源及更新记录 ↗

请按适用版本和实际日志确认原因;本站没有对每一种系统或设备逐项执行文中的操作。