问题原文
原帖链接 · BghinC · CC BY-SA 4.0
I'm building a webapp using Nextjs. As some parts of the app is server side rendering, I have two sitemaps. One generated by Next and the other one generated dynamically.
All pages are successfully added to the search console. However I have also some urls including the bracket slugs like
https://mywebsite.com/[mySlug]
https://mywebsite.com/[mySlug]/aaa
https://mywebsite.com/[myOtherSlug]/bbb
Those urls are not in the sitemaps. Do you have insight on why those urls are present into the search console results?
To be clearer, I have urls like
https://mywebsite.com/my_slug_1
https://mywebsite.com/my_slug_1/bbbb
And they are indexed correctly but the fact that https://mywebsite.com/[mySlug] is indexed is not correct as the bracket slug is just a way to tell Next that the [mySlug] can be any string possible
Thanks!
已采纳解答
原帖链接 · Stephen Ostermiller · CC BY-SA 4.0
Google doesn't rely just on XML sitemaps to know what pages you have on your site and which URLs it should index. See my diatribe on the subject: SEO Myth: Google will only index the pages listed in your XML sitemap.
Google will index URLs that are not in your sitemap if:
- Those URLs have had links either from your own site or from another site. It doesn't matter if the links still exist, once Googlebot finds a page through a link it tends to remember it forever.
- The URLs return 200 OK status with unique content.
If the URLs duplicate other content from your site for which the preferred URL is in your sitemap, Google will usually (but not always) index the URL from your sitemap rather than the URL it finds only through links.
The best way to deal with URLs that you don't want to have indexed it to do one of the following:
- Redirect them to a different URL that you do want indexed to get a different URL with the same content indexed instead.
- Put a
<meta name="robots" content="noindex">tag in place to instruct search engines not to index them. - Password protect them so that Googlebot can't see the contents when it crawls.
- Disallow them in
robots.txtso that Googlebot doesn't even try to crawl them. (Although Google will occasionally index pages it can't crawl, so this method is best for pages that can appear in search results but which you don't want crawlers hitting.) - Return "410 Gone" status to mark them as removed.
Even if the URLs are not indexed, you may see them in various Search Engine Console reports. Search Engine Console reports on all the URLs from your site that Googlebot crawls, not just the ones that are in your sitemap or the ones which Google chooses to index.
摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。
