大网数据广告活动
网站运营与故障问答

How can a search engine crawl a dynamically generated website?

· 52CE 我爱测

A dynamically generated website doesn't have most of its content hard coded on the HTML file. Sometimes it waits for a user to login or wait for an input to generate its views. So it would look something like this. <section id = …

问题原文

原帖链接 · Firdaus Indradhirmaya · CC BY-SA 4.0

A dynamically generated website doesn't have most of its content hard coded on the HTML file. Sometimes it waits for a user to login or wait for an input to generate its views. So it would look something like this.

 <section id = 'blog-content' class="section">
     <div class="columns is-centered">
         <?php echo $content ?>
     </div>
 </section>

In that example, the variable $content contains paragraphs and a title that someone wrote on a certain topic for their blog fetched from their database. How does a crawler know the content of that page if its content is dynamically generated? Since the content is not hard coded, the crawler wouldn't know the content exist because all it would see is the code i post above, right?

EDIT

To expand on what stephen said, the url would be something like this.

www.example.com/blog/$title

The $title variable has the same value as the one in the $content variable above.

已采纳解答

原帖链接 · Maximillian Laumeister · CC BY-SA 4.0

Short answer: That PHP code is run on the server before sending the response to the crawler, so by the time the page reaches the crawler, all that info is already populated.

For sites written using server-side languages such as your example, here's the full lifecycle when a user visits a page:

  1. The user's browser sends an HTTP request to the server for a certain path (such as /an/example/page/).

  2. The server receives the request and determines the appropriate server-side code to run to generate the page. It executes this code, if any (or none if it's a static site).

  3. The server sends the final generated, by that point static HTML page back to the user's browser.

Note that all the code is finished running on the server before the server actually sends any information back to the user's browser (or web crawler).

Things are a little different when the page is generated in part by client-side code (JavaScript) instead, which is a topic for a different discussion.

Regarding waiting for a user to log in or take action, generally search engine crawlers are cookieless and take no user actions, so anything hidden behind a login won't get crawled. Stuff hidden from crawlers behind logins like this is called the deep web, which is a cool term if you ask me.

摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。

资料来源与适用范围

资料来源
Webmasters Stack Exchange
适用范围
以原帖系统、软件版本与发布日期为准(2020-05-23)
内容语言
中文;命令、路径和错误标识保留技术原文
本地收录日期
2026-11-15
授权说明
CC BY-SA 4.0

正文与所需资源由本站本地存储。查看来源及更新记录 ↗

请按适用版本和实际日志确认原因;本站没有对每一种系统或设备逐项执行文中的操作。