大网数据广告活动
网站运营与故障问答

Google can't fetch large sitemap with 50k URLs, nor will browsers render it

· 52CE 我爱测

My sitemap contains 50K URLs/7,8 MB and this following URL syntax: <?xml version="1.0" encoding="UTF-8"?> <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schem…

问题原文

原帖链接 · Eder Leandro · CC BY-SA 4.0

My sitemap contains 50K URLs/7,8 MB and this following URL syntax:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">
<url>
<loc> https://www.ninjogos.com.br/resultados?pesquisa=vestido, maquiagem, </loc> <lastmod> 2019-10-03T17:12:01-03:00 </lastmod>
<priority> 1.00 </priority>
</url>
</urlset>

The problems are:

• Search Console says "Sitemap could not be read";

• The Sitemap takes 1 hour to load and Chrome stops working;

• In firefox the Sitemap has downloaded in 1483ms and fully loaded after 5 mins);

Things I've done without sucess:

• Disable GZip compression;

• Delete my .htaccess file;

• Create a test Sitemap with 1K URLs and the same syntax and sent it to Search Console and it's worked but the 50K URLs Sitemap still shows ""unable to fetch Sitemap";

• Tried to inspect the url directly but it gave error and asks to try again later while the 1K urls worked;

• Tried to validate the Sitemap in five different sites (YANDEX, ETC) and all worked without no error/warning

Any light?

已采纳解答

原帖链接 · Stephen Ostermiller · CC BY-SA 4.0

You should test your sitemap with a downloading program such as curl or wget instead of using a browser like Chrome or Firefox. You should be able to download the file within 3 minutes with a download program. If the file takes longer to download for you, then Googlebot will probably also have problems with it. You can:

  • Upgrade your hosting so your entire site is faster
  • Pre-compress your sitemap with gzip so that the URL is sitemap.xml.gz. That way it will be much smaller and you won't need to disable gzip on your server.
  • Remove lastmod and priority from your sitemap since Google doesn't use them anyway.
  • Break up your sitemap into smaller pieces and use a sitemap index file
  • Remove white space from your sitemap. It looks like all your fields are surrounded by spaces. That isn't correct. It is adding to the size and possibly confusing search engines. <priority> 1.00 </priority> should be <priority>1.00</priority>. Same for <loc> and <lastmod>.
  • Remove unnecessary URLs from your site map.

On the last point, your example is problematic. I believe "resultados pesquisa" translates to "search results". Google doesn't want to have your search results pages indexed. You should be blocking Googlebot from crawling them and you should remove them from your sitemap. See Search results in search results. Having your site search results indexed is bad user experience for users from Google and it can cause Google to penalize your entire site.

You tagged your question seo but your XML sitemap probably won't help your SEO at all. Google doesn't rank pages better because they are in a sitemap, nor will Google usually choose to index a page just because it is in the sitemap. See The Sitemap Paradox. The benefits from having a sitemap are mostly in getting better stats out of Google Search Console. You could also use it as one method of telling Google about your canonical URLs (but they are better ways such as canonical tags.) Because they aren't much use, if your sitemaps are giving you headaches, you can just delete them and not worry about having them. It won't hurt your site or its SEO.

摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。

资料来源与适用范围

资料来源
Webmasters Stack Exchange
适用范围
以原帖系统、软件版本与发布日期为准(2019-10-04)
内容语言
中文;命令、路径和错误标识保留技术原文
本地收录日期
2026-11-15
授权说明
CC BY-SA 4.0

正文与所需资源由本站本地存储。查看来源及更新记录 ↗

请按适用版本和实际日志确认原因;本站没有对每一种系统或设备逐项执行文中的操作。