问题原文
原帖链接 · ntl · CC BY-SA 3.0
robots.txt containts following instructions:
User-agent: *
Allow: /public/
Disallow: /
Sitemap: http://sitedomain.com/sitemapindex.xml
it seems that in this case sitemapindex won't be accessible for the search crawler and should be moved to public folder, is it correct or crawler will process the sitemap in any case?
已采纳解答
原帖链接 · MrWhite · CC BY-SA 3.0
sitemapindexwon't be accessible for the search crawler and should be moved topublicfolder
Yes, that is correct, the XML sitemap won't be crawlable unless moved to the /public/ folder.
Alternatively, you could add an exception for your sitemap:
Allow: /sitemapindex.xml
摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。
