问题原文
原帖链接 · Ivan Balepin · CC BY-SA 4.0
I would like to not have OpenAI, ChatGPT or the potentially associated Bingbot scrape my website. The latter seems to have a defined bot name. What about the first two? They seem to be using Common Crawl scraper among other sources, so that's another one to block in robots.txt, with what name though? Any other items to add to robots.txt to ban OpenAI? Is it known if they honor robots.txt or like (e.g.) rogue Linkedin scraping just go and do it all rules be damned? Earlier discussions on this topic seem inconclusive.
社区高分解答
原帖链接 · Maniac · CC BY-SA 4.0
Yes, it is possible both through a robots.txt declaration and to be restricted at the IP address level.
If you want to restrict it via robots.txt, you need the following declarations:
User-agent: ChatGPT-User
Disallow: /
If you want to block it at the IP address level, you need to block the following IP address: 23.98.142.176/28
More information could be found here: ChatGPT-User
摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。
