大网数据广告活动
网站运营与故障问答

Is it possible to exclude just OpenAI - ChatGPT from scraping my website?

· 52CE 我爱测

I would like to not have OpenAI, ChatGPT or the potentially associated Bingbot scrape my website. The latter seems to have a defined bot name. What about the first two? They seem to be using Common Crawl scraper among other sourc…

问题原文

原帖链接 · Ivan Balepin · CC BY-SA 4.0

I would like to not have OpenAI, ChatGPT or the potentially associated Bingbot scrape my website. The latter seems to have a defined bot name. What about the first two? They seem to be using Common Crawl scraper among other sources, so that's another one to block in robots.txt, with what name though? Any other items to add to robots.txt to ban OpenAI? Is it known if they honor robots.txt or like (e.g.) rogue Linkedin scraping just go and do it all rules be damned? Earlier discussions on this topic seem inconclusive.

社区高分解答

原帖链接 · Maniac · CC BY-SA 4.0

Yes, it is possible both through a robots.txt declaration and to be restricted at the IP address level.

If you want to restrict it via robots.txt, you need the following declarations:

User-agent: ChatGPT-User
Disallow: /

If you want to block it at the IP address level, you need to block the following IP address: 23.98.142.176/28

More information could be found here: ChatGPT-User

摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。

资料来源与适用范围

资料来源
Webmasters Stack Exchange
适用范围
以原帖系统、软件版本与发布日期为准(2023-05-13)
内容语言
中文;命令、路径和错误标识保留技术原文
本地收录日期
2026-11-15
授权说明
CC BY-SA 4.0

正文与所需资源由本站本地存储。查看来源及更新记录 ↗

请按适用版本和实际日志确认原因;本站没有对每一种系统或设备逐项执行文中的操作。