Crawler Catalog
1,876 known bots, crawlers, and AI agents — with UA patterns, verification methods, and recommended access policies.
Not sure what "crawler" vs "scraper" vs "agent" means here? See our terminology guide →
75 results · AI Coding Agent
2IP
2ip
Cloudflare
AgentReadinessScanner
SmartBear
Alertsite
Stack Moxie
AutomatonHeadlessChrome
Better Stack
Better
Binary Canary
Chirp
Anthropic
Claude-Code
GitHub
Code
CrazyEgg
Crazy
Critical CSS
Criticalcss.com
xAI
Cursor
Datadog
Datadog
Dr. Link Check
DLC
Dotcom-Monitor
DMBrowser
Electronic Frontier Foundation
Do
Doctom
Drata
DreamHost
DreamHost
DubBot
Dubbotbot
Electronic Frontier
EFF
elmah.io
elmah.io
Foregenix
Foregenix
Freshworks
FreshpingBot
Microsoft / GitHub
Google-Gemini-CLI
Google-Structured-Data-Testing-Tool
GoogleAssociationService
Grafana Labs
grafana
Grafana Labs
Grafana's
Honeybadger
Honeybadger
Jagged
Automattic
jetmon
Let's Encrypt
Let's
360 Monitoring
monitoring360
360 Monitoring
monitoring360bot
New Relic
New
New Relic
NewRelicbot
NitroPack
Nitro-Optimizer-Agent
NixStats
NIXStatsbot
Noibu
Noibu
Oh Dear
Oh
Oh Dear
OhDear.app
OnlineOrNot
OnlineOrNot.com_bot
Anomaly
opencode
Parse.ly
parsely.com
SolarWinds
Pingdom.com_bot
PingPing
pingping.io
ProjectShield
Paessler
PRTGCloudBot
Pulsetic
pulsetic.com
Microsoft
PWABuilderHttpAgent
Scrunch
Scrunchbot
Sentry
SentryUptimeBot
StackPath
Server
Silktide
Silktide
Swifteq
SwifteqLinkChecker
Grafana Labs
synthetic-monitoring-agent
T3Versions
t3versionsBot
Testomato
Testomatobot
Thousand Eyes
Thousand