问题原文
原帖链接 · space and time · CC BY-SA 3.0
Some bots have been crawling my site for every link that ends with:
?utm_source=dlvr.it&utm_medium=twitter
I haven't checked out its IP.
Then other bots (10+) follow the link rules ?utm_source=dlvr.it&utm_medium=twitter and also crawl my site. This results in a huge amount of traffic which then causes my site to shut down. I have added 10+ other bots into my blacklist with an HTTP 403 status code when they access my site.
But I think the best way is to find out the first bot which crawled my site for every link that ends with:
?utm_source=dlvr.it&utm_medium=twitter
Or, use a an HTTP 403 redirect status code when the URL contains:
?utm_source=dlvr.it&utm_medium=twitter
I know of some ways to add .htaccess code that would prevent someone from crawling my xmlrpc.php page, such as:
<Files xmlrpc.php>
Order Deny,Allow
Deny from all
</Files>
But what about a query in URL?
已采纳解答
原帖链接 · Mike -- No longer here · CC BY-SA 3.0
If you have the mod_rewrite module installed, then you can put this in your .htaccess file in the root folder of your website (which usually is the public_html folder):
RewriteEngine On
RewriteRule ^\?utm_source\=dlvr.it\&utm_medium\=twitter$ - [R=403,NC,L]
You might have to remove the \ from the =, I can't remember if equals needs escaping.
Another way would be this if you're searching for the string anywhere in the URL:
RewriteEngine On
RewriteRule ^(.*)\?utm_source\=dlvr.it\&utm_medium\=twitter(.*)$ - [R=403,NC,L]
The NC at the end means not case-sensitive, so if the text is all upper-case, then the bots would be directed to an error 403.
摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。
