问题原文
原帖链接 · Stickers · CC BY-SA 3.0
I would like to allow folder /news/ and disallow all the sub folders under /news/ e.g. /news/abc/, /news/123/. How can I do that please?
I think Disallow: /news/ will block everything in it, including /news/ itself.
Will Disallow: /news/*/ do the job, since there is no easy way to test it, I want to make sure.
已采纳解答
原帖链接 · Henry Visotski · CC BY-SA 3.0
User-agent: *
Allow: /news/$
Disallow: /news/
Explanation:
Google's robots.txt spec (https://developers.google.com/search/reference/robots_txt), which is more up to date than the "official" spec, states that:
/fish/ will match anything in the /fish/ folder but will not match /fish (and, no wildcard necessary, since "The trailing slash means this matches anything in this folder.") If you kinda reverse engineer that:
User-agent: * (or whatever user agent you want to talk to)
Allow: /news/$ (allows /news/ but the $ character says the allow can't go beyond /news/)
Disallow: /news/ (disallows anything in the /news/ folder)
Test it in Google Search Console, or in Yandex (https://webmaster.yandex.com/tools/robotstxt/) to ensure it works for your site.
摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。
