Claude
recommended: observe
Operated by Anthropic
ai crawler
ClaudeBot helps enhance the utility and safety of generative AI models by collecting web content that could potentially contribute to their training.
How to identify it
User-agent contains any of:
ClaudeBot
robots.txt
Well-behaved crawlers honor robots.txt. To control Claude:
Allow full access
User-agent: ClaudeBot
Allow: /Block entirely
User-agent: ClaudeBot
Disallow: /
Grey-area scrapers ignore robots.txt. Botscope enforces your policy at the edge or origin regardless —
see how →
See Claude on your own site
Botscope shows every bot and AI agent hitting your site — and lets you allow, challenge, or block each one. Observe first, enforce when you're ready.
Start free — connect in minutes