大网数据广告活动
网站运营与故障问答

"URL is unknown to Google" error on a "new" forum site

· 52CE 我爱测

Summary: We've recently made certain sections of our (previously) private forum and community open so that users can access areas of the site without registering. The site, however, is encountering indexing issues... The main pag…

问题原文

原帖链接 · Hassan Uddeen · CC BY-SA 4.0

Summary:

We've recently made certain sections of our (previously) private forum and community open so that users can access areas of the site without registering. The site, however, is encountering indexing issues...

The main page home page is being indexed and is discoverable via search. However, the other key pages, forum pages especially, aren't being indexed or crawled - and are potentially being blocked by our site privacy policy.

The site has been created using our learning management system.

(GSC live test screenshot of homepage)

Naturally, I requested Google to index certain pages via GSC, and I haven't heard anything back for more than two weeks.

Info on the URLs:

These are forum threads or "/discussion/" pages that return the "URL is unknown status to Google" status and are not being crawled or indexed:

https://example.com/mod/forum/discuss.php?d=25132

I just tested the live URL for these pages, and the live test page screenshot from GSC is showing our privacy policy instead of the actual page!

It seems like the privacy policy page has some sitewide robot.txt or CORS issues that are the root cause? (Again, I'm not super technical and this is really confusing me now!)

What do you guys think is the root cause of these indexing issues?

Other potential causes and contributors, but I'm betting on the problem being the site policy:

  • No sitemap to help Google determine what the important pages are— although Google can crawl and index pages without there being a sitemap.

  • Canonical URLs - by default, none of the pages have self-referencing canonical URL tags, probably related to the configuration of the site/platform.

  • Many of the pages have the same </ H1 > although I can’t see why this would trigger crawl and index issues.

I've asked our team/devs who deal with hosting settings for the site to check if something could be preventing Googlebot from crawling and indexing the site properly (although I'm not sure if this requires a log file analysis or if it’s a simple act of checking firewall-related settings etc.)?

已采纳解答

原帖链接 · Maximillian Laumeister · CC BY-SA 4.0

I believe the issue here is a lot simpler than you're expecting. It appears that your website is 303-redirecting users to your privacy policy page if they haven't accepted the privacy policy yet.

Googlebot can't accept the privacy policy and doesn't have any memory across page visits, it just follows the redirect every time. So all Googlebot sees is your privacy policy, for every discussion page on your site.

To get these pages indexed, you'll need to prevent your site from redirecting all of your pages to your privacy policy page for new users. Perhaps you can allow users to view your content without accepting the policy, and only require acceptance when they create an account or otherwise go to contribute?

摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。

资料来源与适用范围

资料来源
Webmasters Stack Exchange
适用范围
以原帖系统、软件版本与发布日期为准(2021-10-12)
内容语言
中文;命令、路径和错误标识保留技术原文
本地收录日期
2026-11-15
授权说明
CC BY-SA 4.0

正文与所需资源由本站本地存储。查看来源及更新记录 ↗

请按适用版本和实际日志确认原因;本站没有对每一种系统或设备逐项执行文中的操作。