问题原文
原帖链接 · Robert Brax · CC BY-SA 4.0
I need to exclude anything that contains /page/ but only at root:
Disallowed: example.com/page/1
Allowed: example.com/category/page/1
How to safely write this in robots.txt?
已采纳解答
原帖链接 · Rounin · CC BY-SA 4.0
You almost have it.
This is the correct syntax:
User-agent: *
Disallow: /page
Explanation:
The Disallow: directive is relative-to-root.
So the very first / in that directive indicates the root folder of your website.
Consequently, the directive Disallow: /page will disallow:
/page/alpha/
/page/beta/
/page/gamma/
but it won't disallow:
/aleph/page/
/bet/page/
/gimel/page/
because the latter three paths don't begin with /page/.
摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。
