大网数据广告活动
网站运营与故障问答

What is the correct way to write my "robots.txt" file?

· 52CE 我爱测

I have written the following code inside my robots.txt file: User-Agent: Googlebot Disallow: User-agent: Mediapartners-Google Disallow: Sitemap: http://example.com/sitemap.xml Is my robots.txt is correct? I only want two user age…

问题原文

原帖链接 · ashutosh · CC BY-SA 3.0

I have written the following code inside my robots.txt file:

   User-Agent: Googlebot
   Disallow:

   User-agent: Mediapartners-Google
   Disallow:

   Sitemap: http://example.com/sitemap.xml

Is my robots.txt is correct? I only want two user agent to follow my site (i.e., Googlebot & Mediapartners-Google).

已采纳解答

原帖链接 · dan · CC BY-SA 3.0

Almost, see the bottom of the section named, "Blocking user-agents" in the following: Google Webmaster Tools: Block or remove pages using a robots.txt file

According that, you should have:

User-agent: *
Disallow: /

User-agent: Googlebot
Allow: /

User-agent: Mediapartners-Google
Allow: /

sitemap: http://example.com/sitemap.xml

See the bottom of the page in the above link for how to test your robots.txt file.

摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。

资料来源与适用范围

资料来源
Webmasters Stack Exchange
适用范围
以原帖系统、软件版本与发布日期为准(2013-06-20)
内容语言
中文;命令、路径和错误标识保留技术原文
本地收录日期
2026-11-15
授权说明
CC BY-SA 3.0

正文与所需资源由本站本地存储。查看来源及更新记录 ↗

请按适用版本和实际日志确认原因;本站没有对每一种系统或设备逐项执行文中的操作。