大网数据广告活动
网站运营与故障问答

Page cannot be crawled by Google at all

· 52CE 我爱测

I am trying to index a page which was set to "discourage search engines to index" until 2021-10-31. Now it's 2021-11-06 and Google does not even try to access the page. If I use the search console, it get errors it cannot be fetc…

问题原文

原帖链接 · Alex · CC BY-SA 4.0

I am trying to index a page which was set to "discourage search engines to index" until 2021-10-31. Now it's 2021-11-06 and Google does not even try to access the page.

If I use the search console, it get errors it cannot be fetched. Same with the mobile testing tool:

My page meta is content:

<meta name='robots' content='max-image-preview:large' />

https://page.et/robots.txt

User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php

Sitemap: https://example.com/wp-sitemap.xml
  • There was a forward from page.et/robots.txt to page.et/en/robots.txt which I also disabled. In my Chrome it's still cached, but using an incognito tab I can see the robots.txt

  • When trying to request the page in Google I see zero evidence of Google trying to access my page, not even the robots.txt - in my web server logs.

Is Google really blocking all requests to my page and storing this persistently for more then 4 days ? Or is there maybe really a technical reasons it does not access my page?

Are there any additional tools to debug this?

Update: In the search console, it displays this:

How do I read these reasons?

Edit 2021-11-07 Same result .. I don't think it's just a caching problem by Google :-( I mostly wonder how I can debug this properly. The output by Google search console does not seem helpful :-(

Edit: I am getting some strange errors in Google Pagespeed which I tried to see if Google can access the page at all. Following up eventual server configuration errors here: https://serverfault.com/questions/1082838/pagespeed-cannot-always-access-page-failed-document-request-ssl-certification

已采纳解答

原帖链接 · Roman Mikhol · CC BY-SA 4.0

.Et domain zone NS block Googlebot

The problem is on the side of the .et name servers - they "block" Googlebot.

This Google Support thread reports a problem with indexation with all domains in ".et" zone

摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。

资料来源与适用范围

资料来源
Webmasters Stack Exchange
适用范围
以原帖系统、软件版本与发布日期为准(2021-11-05)
内容语言
中文;命令、路径和错误标识保留技术原文
本地收录日期
2026-11-15
授权说明
CC BY-SA 4.0

正文与所需资源由本站本地存储。查看来源及更新记录 ↗

请按适用版本和实际日志确认原因;本站没有对每一种系统或设备逐项执行文中的操作。