Skip to main content

Solutions

AI coding agents on your docs

Coding assistants fetch documentation on a developer’s behalf, often at machine pace. See which ones reach your docs and decide what each is allowed to read.

Start free Browse the catalogue

Free plan, no card. 100,000 events a month with every feature on.

botscope.io · last 7d

This site, classified by this product. Nothing here is a mock-up.

live
requests
4,245
human
80%
AI agents
0%
stopped
0%
verdict stream
  1. signals
    • threat_feed: datacenter — fail (listed on a subscribed feed)
    • anomaly_score: 0.25 — pass (combined disagreement across the collected signals)
  2. signals
    • threat_feed: datacenter — fail (listed on a subscribed feed)
    • anomaly_score: 0.25 — pass (combined disagreement across the collected signals)
  3. signals
    • anomaly_score: 0.40 — pass (combined disagreement across the collected signals)

botscope.io's own traffic only — never a customer's. No addresses, no query strings, and a path is shown only when it is already a public page.

In short

  • Coding assistants fetch documentation while a developer works, which is a read on behalf of a person, not a crawl.
  • They arrive at machine pace and mostly do not appear in analytics as anything identifiable.
  • Botscope names them separately from training collectors, because the decision about them is different.
  • For most API documentation, allowing them is the point — but it should be a decision, not a default nobody made.

Updated 16 September 2026

01 · a read with a person behind it

A read with a person behind it

A coding agent fetching your API reference is not collecting a dataset. Someone is building against your product right now and the agent is reading the manual for them.

That makes the usual scraper argument fit badly. The traffic is automated, unattributed and can be bursty — and it is also a developer in the middle of an integration, which is the audience documentation exists for.

The practical problem is that you cannot weigh any of this without first being able to see it.

02 · coding agents in the catalogue

Coding agents in the catalogue

Bot Operator Default policy Verifiable
2ip bot 2IP Observe By user agent
AgentReadinessScanner Cloudflare Observe By user agent
AutomatonHeadlessChrome Stack Moxie Observe By user agent
Bugsnag script fetcher Bugsnag Observe By user agent
Chirp Binary Canary Observe By user agent
Claude-Code Anthropic Observe By user agent
Code GitHub Observe By user agent
Crazy Egg CrazyEgg Observe By user agent
Criticalcss.com Critical CSS Observe By user agent
Cursor xAI Observe By user agent

Read live from the Botscope crawler catalogue. Default policy is the catalogue default; every agent can be overridden per site.

03 · what to decide

What to decide

  1. 1 Documentation and API reference: usually allow. This is a developer reading your docs through a tool.
  2. 2 Pricing, changelog, status: usually allow, for the same reason.
  3. 3 Application endpoints and anything behind a session: these are not documentation, and an agent has no reason to be there.
  4. 4 Watch the rate rather than the identity. An agent that fetches a hundred pages a second is a load question, not a policy question.

04 · what is not knowable

What is not knowable

Which human asked, and what they were building. The agent identifies itself; the person behind it does not, and nothing at the network layer changes that.

Questions

Should I block coding agents?

On documentation, rarely — the traffic is a developer evaluating your product. On application endpoints, there is no reason for an agent to be there at all.

Do they respect robots.txt?

It varies by operator, and the catalogue records what each one declares. A fetch made because a person asked for that specific page is treated differently by several operators than a crawl.

Will this show up in my analytics?

Generally not in a way you can identify, which is the reason for classifying at the origin instead.

Can I rate-limit rather than block?

Yes. Challenge and rate policies are per agent, so an agent can stay allowed while capped.

See which of these reach your site

Free plan, no card: 100,000 events a month with every feature on. Nothing is blocked until you turn enforcement on.

Start free Browse the catalogue

Related