大网数据广告活动
网站运营与故障问答

What does "Disallow: /search" mean in robots.txt?

· 52CE 我爱测

In my blog's Google Webmaster Tools panel, I found the following code in my robots.txt of blocked URLs section. User-agent: Mediapartners-Google Disallow: /search Allow: / I know that Disallow will prevent Googlebot from indexing…

问题原文

原帖链接 · Sathiya Kumar V M · CC BY-SA 3.0

In my blog's Google Webmaster Tools panel, I found the following code in my robots.txt of blocked URLs section.

User-agent: Mediapartners-Google
Disallow: /search
Allow: /

I know that Disallow will prevent Googlebot from indexing a webpage, but I don't understand the usage of Disallow: /search.

What is the exact meaning of Disallow: /search?

已采纳解答

原帖链接 · unor · CC BY-SA 3.0

In the Disallow field you specify the beginning of URL paths of URLs that should be blocked.

So if you have Disallow: /, it blocks everything, as every URL path starts with /.

If you have Disallow: /a, it blocks all URLs whose paths begin with /a. That could be /a.html, /a/b/c/hello, or /about.

In the same sense, if you have Disallow: /search, it blocks all URLs whose paths begin with the string /search. So it would block the following URLs, for example (if the robots.txt is in http://example.com/):

  • http://example.com/search
  • http://example.com/search.html
  • http://example.com/searchengine
  • http://example.com/search/
  • http://example.com/search/index.html

While the following URLs would still be allowed:

  • http://example.com/foo/search
  • http://example.com/sea

Note that robots.txt doesn’t know/bother if the string matches a directory, a file or nothing at all. It only looks at the characters in the URL.

摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。

资料来源与适用范围

资料来源
Webmasters Stack Exchange
适用范围
以原帖系统、软件版本与发布日期为准(2013-07-09)
内容语言
中文;命令、路径和错误标识保留技术原文
本地收录日期
2026-11-15
授权说明
CC BY-SA 3.0

正文与所需资源由本站本地存储。查看来源及更新记录 ↗

请按适用版本和实际日志确认原因;本站没有对每一种系统或设备逐项执行文中的操作。