大网数据广告活动
网站运营与故障问答

Google keeps crawling pages that don't exist

· 52CE 我爱测

I made a simple new website which went live about a month and a half ago (replacing the last one). The site contains new pages, like for example the navigation markup: <div class="header"> <div class="navigation menu"> <ul> <li><…

问题原文

原帖链接 · Vucko · CC BY-SA 3.0

I made a simple new website which went live about a month and a half ago (replacing the last one). The site contains new pages, like for example the navigation markup:

<div class="header">
    <div class="navigation menu">
        <ul>
            <li><a href="index.php">Home page</a></li>
            <li><a href="products.php">Products</a></li>
            <li><a href="reference.php">Reference</a></li>
            <li><a href="about_us.php">About us</a></li>
            <li><a href="contact.php">Contact</a></li>
        </ul>
    </div>
</div>

The problem is, that google crawlers keep looking for the old pages, which I removed and aren't on the server.

I keep removing the crawl errors (marked as fixed) in the webmaster tools, but the crawlers keep attempting to crawl those pages.

I believe that the last website version is cached somewhere, so is there a way to remove it? (I don't have a lot of experience with google webmaster tools.)

已采纳解答

原帖链接 · MrWhite · CC BY-SA 3.0

This is going to happen for a looong time. Other sites might be linking to the old URLs, which will prompt Google to crawl them and/or your site might just be having problems (in Googles eyes) and is giving you the benefit of the doubt that the pages might return. Either way, Google continues to crawl old pages for a long time. It would be far worse for Google to suddenly stop crawling your pages after getting a bunch of 404's.

If the pages genuinely do not exist and there is no alternative then it is correct to return a 404 (Not Found). Or you can return a 410 (Gone) instead for these pages that are never going to return - that is a far stronger/definite indication to Google that the pages aren't coming back.

Note that serving a 404 (or 410) for these pages, and having them reported as such in GWT is not necessarily a bad thing, and will not detrimentally affect your site in itself. This is a private report for your benefit.

However, what can be bad for SEO is if there are other sites linking to your old page that would have otherwise passed PageRank. By returning a 404, these sites will no longer be able to pass that PageRank. If you have alternative pages for those removed, then 301 redirect to the new location in order to preserve your ranking, help search engines re-index your content and ... to please your users. But if you have simply removed the old content and not replaced it then you need to be prepared to take the potential SEO hit.

摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。

资料来源与适用范围

资料来源
Webmasters Stack Exchange
适用范围
以原帖系统、软件版本与发布日期为准(2013-08-17)
内容语言
中文;命令、路径和错误标识保留技术原文
本地收录日期
2026-11-15
授权说明
CC BY-SA 3.0

正文与所需资源由本站本地存储。查看来源及更新记录 ↗

请按适用版本和实际日志确认原因;本站没有对每一种系统或设备逐项执行文中的操作。