Cloudflare adds continuous checks for Verified bots: a trust band based on whether they honor robots.txt and slow down after a 429
After proving identity, crawlers and agents now have to keep earning trust through behavior.
// Key points
- Verified bots must identify themselves accurately and securely, do only what they say, and not be abusive; the new system evaluates the third rule.
- The first signals are whether observed traffic stays consistent with robots.txt over time and whether a bot slows down or pauses after an HTTP 429.
- Results become an explainable trust band built from sustained patterns rather than one-off events, and Cloudflare plans to publish the scores on Radar.
- It is used internally for now and will open to all Cloudflare customers, who will choose the trust threshold for handling Verified bot traffic.
Builder's takeIf you build browser agents or crawlers, this matters more than any model launch: getting through will depend on a behavior record, not just a signature. I’d check our crawling code reads robots.txt and backs off exponentially on a 429; that used to be courtesy, now it’s scored. Read with the Cloudflare and OpenAI discovery pilot on 10-16, Cloudflare is making itself the rule-setter for AI traffic.
// Background · from #AI agents
Full timeline →- Oct 17 AWS shares the evaluation loop behind its internal sales agents: three months took the pass rate from 93% to 96% and pure hallucinations from 138 to under 52 per 10,000 responses
- Oct 16 NVIDIA opens the 120B Nemotron 3.5 Super VL with datasets and training recipes; the 74B-A9B Bolt fits in 24GB with native 2-bit quantization
- Oct 15 Claude API launches OpenTelemetry export in beta to trace Messages API requests and Managed Agents sessions