Skip to main content
Docs

Botscope in Claude

The Botscope connector lets Claude answer questions about crawlers and AI agents from the Botscope catalogue instead of from memory. The catalogue tools work without a Botscope account; with one, Claude can also read aggregated bot traffic and agent policies for your own sites.

Install

Add it from the Claude directory, or as a custom connector with the URL https://mcp.botscope.io/mcp. In Claude Code: claude mcp add --transport http botscope https://mcp.botscope.io/mcp.

Tools

Every tool is read-only except set_agent_policy, which changes one agent's policy on one of your sites.

Catalogue — no account needed

lookup_crawler
The catalogue record of a bot by slug: operator, category, how it can be verified, its default policy and a link to the record.
search_crawlers
Search by name, slug, operator or robots.txt token. Up to 15 records.
identify_crawler
Which record a User-Agent string claims to be, and — if you give an IP — whether that IP is inside the record's published IP ranges. A User-Agent match alone is reported as an unverified claim. No reverse DNS or signature check is performed.
robots_policy
A robots.txt fragment for one of three intents: block training crawlers, block all AI agents, or follow catalogue defaults. At most 80 User-agent groups. robots.txt is a hint to crawlers; it does not enforce anything.

Your sites — after signing in

list_sites
Your sites: id, name, domain, and whether each is in observe or enforce mode. The shared demo site is never included.
traffic_summary
Aggregated traffic for 1 to 28 days: bot versus human share, categories, number of distinct agents, top AI vendors and top agents.
list_agent_hits
Up to 25 agents seen on the site, each with hit count and the policy the site applies to it.
get_agent_policy
The policy for one agent: catalogue default, effective action, whether the catalogue pins it, and the site mode.
get_site_robots_txt
How your generated robots.txt is composed; the file text only when you ask for it.
set_agent_policy
Sets one agent to allow, observe, challenge or block on one site. Needs the separate "change policies" permission and a site role that can manage protection. Refused for agents the catalogue pins. Each change is written to the site's audit log. On a site in observe mode the policy is saved but nothing is refused until the site enforces — the tool says so in its answer.

Signing in

Claude asks you to sign in the first time you ask about your own site, not when you install the connector. You sign in on botscope.io with your existing account and choose what to allow: listing your sites, reading traffic summaries, reading policies, and — separately — changing agent policies. You can connect for reading only. Access tokens last an hour and are renewed for up to 30 days. There is no page in the panel to revoke a connection yet; contact us and we will revoke it.

What it reads, and what it does not

  • It reads the public catalogue only — the same data as the crawler pages on this site.
  • Without signing in it reads no site data at all. After signing in it reads only sites in organizations you belong to, and only what you allowed.
  • Site answers are aggregates. They never contain visitor IP addresses, query strings, visitor ids or raw events, and never your site API keys.
  • It can change one agent's policy on one site, and only with the permission you granted — nothing else. It cannot change site modes, rules, bans, feeds or keys, and it has no bulk change.
  • Responses never include the cached IP range lists, signing keys or upstream source data.
  • Catalogue descriptions come from external feeds. They are cleaned and shortened before they reach Claude, and they are presented as card text.

What is sent to mcp.botscope.io

Only the parameters of the tool Claude calls: a slug, a search term, a User-Agent string with an optional IP, a robots.txt intent, or a site id and period — plus a hash of the connector session, used for rate limits, and your access token when you are signed in. Not your conversation, and not who you are. What you asked is not written to our application log; the web server access log records request paths, including search terms, under its normal retention.

Limits and errors

Each conversation can make 60 lookups, 30 searches, 30 identifications and 20 robots.txt drafts a minute. Past that, the tool returns an error saying when to try again. An unknown slug or an invalid argument also returns an error with a sentence explaining it, never an empty result.

Support

Questions or a wrong catalogue entry: contact us.