大网数据广告活动
网站运营与故障问答

How can I get a web crawler to see dynamic data injected by JavaScript in 2019?

· 52CE 我爱测

Originally posted this on stackoverflow and it got put as off-topic so posting here. I recently read this google webmasters article from 2015 on accessibility and SEO for web pages using AJAX. It states the deprecation of their o…

问题原文

原帖链接 · Timothy Fisher · CC BY-SA 4.0

Originally posted this on stackoverflow and it got put as off-topic so posting here.

I recently read this google webmasters article from 2015 on accessibility and SEO for web pages using AJAX. It states the deprecation of their old crawling scheme using the hashbang method, and then states the updated proposal:

Times have changed. Today, as long as you're not blocking Googlebot from crawling your JavaScript or CSS files, we are generally able to render and understand your web pages like modern browsers. To reflect this improvement, we recently updated our technical Webmaster Guidelines to recommend against disallowing Googlebot from crawling your site's CSS or JS files.

Any research I've done on this topic has led to old articles that point to and reference the deprecated article/method. So, consider this simple data binding snippet (not AJAX just plain JSON):

https://codepen.io/tfisher9180/pen/qGRbVd

The select box populates a table from JSON data.

Obviously the source of the page contains no table data. So, what is the recommend approach to expose each of the option's table data to a web crawler?

已采纳解答

原帖链接 · Stephen Ostermiller · CC BY-SA 4.0

Googlebot does know how to execute JavaScript these days, however there are limitations:

  • Googlebot renders pages sometime after crawling them. It could take extra months for Google to get around to rendering and find extra content written into the page by JavaScript.
  • Googlebot doesn't click on anything. Any content that is only shown after user interaction doesn't get indexed. (Googlebot does however still find links in dynamic content and will later crawl and render linked pages.)

In your example, the content is only shown after a user interacts with a select box on the page. That means that Googlebot will never see it. You have two options:

  1. Write all the data into the page on page load. Rather than use a select box to get to the different areas of content, let the user scroll to them.
  2. Create separate pages and separate URLs for different content. Instead of a select box to change the content, use links. You could even put the links in a pull down menu that looks and acts a lot like a select box. For users, you could intercept the link clicks, show the content in the current page and use pushState to change the page URL without actually loading a new page. Just make sure that when the page is loaded with that different URL that the appropriate content is shown by default.

Your example doesn't use AJAX, but it doesn't change too much to use AJAX. Googlebot now downloads supporting resources such as external JavaScript, CSS, and AJAX when it renders pages. Again that AJAX download has to be triggered during the page load, and not as a result of user interaction. Googlebot uses a fairly short timeout when fetching resources, so your server has to be able to return them to Googlebot relatively quickly when Googlebot tries to fetch them. Other than that, Googlebot treats inline JS data almost the same as AJAX fetched data.

You might be interested in watching this video that Google just released that goes through a lot of this. Google Webmaster Analytist Martin Splitt has an entire video series on the topic.

摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。

资料来源与适用范围

资料来源
Webmasters Stack Exchange
适用范围
以原帖系统、软件版本与发布日期为准(2019-05-14)
内容语言
中文;命令、路径和错误标识保留技术原文
本地收录日期
2026-11-15
授权说明
CC BY-SA 4.0

正文与所需资源由本站本地存储。查看来源及更新记录 ↗

请按适用版本和实际日志确认原因;本站没有对每一种系统或设备逐项执行文中的操作。