问题原文
原帖链接 · Déjà vu · CC BY-SA 4.0
Searching for robots.txt pattern matching rules (inc. the non official Google rules), I found this Google dev page where it says
the
/fishrule matches any path that starts with/fish(...) it doesn't match the path/desert/fish
but they also say
the
/fish/rule matches anything in the/fish/folder (...) it matches the path/animals/fish/
That means that as long as a path is defined within the rule (like /something/), a match occurs wherever that pattern is found within the URL path. But if the rule doesn't define a path, having a leading / but no trailing /, the path starts from root?
(regexes would be something like
/fish⇨^/fish.*/fish/⇨^.*/fish/.*
the .* and the second ^ are not necessary, it's just to emphasize the regex patterns)
That sounds quite counter-intuitive to me (I thought that could be a mistake in the Google doc). Can someone confirm this understanding of the syntax?
已采纳解答
原帖链接 · MrWhite · CC BY-SA 4.0
the
/fish/rule matches anything in the/fish/folder (...) it matches the path/animals/fish/
This does look like a mistake in the Google docs IMO (as you suggest) for several reasons:
That example does not work in Google's own robots.txt Tester:
That would seem to be the only example (AFAICS) that does not follow the expected prefix-matching nature of
robots.txtpattern matching. Patterns always match from the document root (from the left). So this is indeed counter-intuitive.If you needed to match
/fish/anywhere in the URL-path then you can use an explicit wildcard, eg./*/fish/(or even*/fish/). So having/fish/also match anywhere is not required and again counter-intuitive.If the pattern
/fish/matches anywhere then you are going to have to always have rules that negate matching/fish/in other parts of the URL. This would create more work and is counter-intuitive. If for instance you hadDisallow: /fish/and I link to your homepage with/?/fish/then I've effectively created a nofollow link that can be shared. (This should not be possible, at least not so easily/accidentally.)
摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。
