OpenAI、谷歌、微软、Anthropic 支持新爬虫退出标准
原标题：The biggest names in AI are backing a new way to opt out of scraping
AI 摘要
OpenAI、Google、Microsoft 和 Anthropic 宣布将采用一项由 Anthropic 标准技术负责人 Kevin Kelley 在 IETF 提出的共享「policy token」信号，该信号置于网站 robots.txt 中，让发布者一次性向多家 AI 公司表达其内容是否可用于 AI 训练，而无需逐家设置。此举回应了发布者面对大量 AI 爬虫抓取内容的压力，此前已有 RSL 收费标准和 Cloudflare 默认拦截等做法。Anthropic 已采用该信号，Google 和 Microsoft 计划明年支持，OpenAI 称将在未来数月内实施；Microsoft 还将提供区分 AI 训练与 AI grounding 的更细粒度控制，且不影响网站在 Bing 搜索结果中的呈现。
正文节选
OpenAI, Google, Microsoft, and Anthropic say they will adopt a new standard that publishers can use to opt out of being scraped by AI, as reported earlier by Search Engine Roundtable. With this control, publishers can apply their AI training preferences across multiple AI companies, instead of setting them individually. The biggest names in AI are backing a new way to opt out of scraping The new control will allow publishers to apply their AI training preferences across multiple crawlers. The sh