大网数据广告活动
网站运营与故障问答

Will Googlebot click a button/link to reveal AJAX content?

· 52CE 我爱测

my client has to inform it's customers about some new regulations that the Googlebot should NOT crawl. It is not possible to place this information on a separate page and disallow Google to crawl it. So the idea is to place a but…

问题原文

原帖链接 · pixelbash · CC BY-SA 4.0

my client has to inform it's customers about some new regulations that the Googlebot should NOT crawl. It is not possible to place this information on a separate page and disallow Google to crawl it. So the idea is to place a button/link on the page, that will AJAX-load the corresponding information only when the user clicks it. My assumption is, that Google is unable to click the link and crawl that specific AJAX content.

Am I right? And if yes, is there an official documentation that proofs my point on this?

已采纳解答

原帖链接 · Stephen Ostermiller · CC BY-SA 4.0

Googlebot does not interact with the page like a user. It does not click on anything. It does not scroll. If content is loaded into the page when users click, Google is not going to index it as part of that page.

However, Googlebot still may find AJAX content to which users have to click. That is because Googlebot scans the page source, the rendered document object model, and loaded JavaScript files for things that look like links. Googlebot is likely to discover the URL for the AJAX that way and then Googlebot may crawl it.

The only reliable way to keep Googlebot from crawling something is to use robots.txt. You could put the AJAX URL itself into robots.txt. Googlebot would still be allowed to crawl the page, but would not be allowed to load the AJAX. So you could use Disallow: /regulations.json in robots.txt but still have Googlebot crawl and index /mypage.html which calls regulations.json via AJAX.

You can prevent Googlebot from seeing portions of the page like this. It doesn't matter if the content is loaded via AJAX on click, or on document load. If Googlebot isn't allowed to crawl the AJAX URL, Google won't see the content. See Preventing robots from crawling specific part of a page for full details.

摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。

资料来源与适用范围

资料来源
Webmasters Stack Exchange
适用范围
以原帖系统、软件版本与发布日期为准(2020-07-01)
内容语言
中文;命令、路径和错误标识保留技术原文
本地收录日期
2026-11-15
授权说明
CC BY-SA 4.0

正文与所需资源由本站本地存储。查看来源及更新记录 ↗

请按适用版本和实际日志确认原因;本站没有对每一种系统或设备逐项执行文中的操作。