Updated daily
Crawler Catalog
1,876 known bots, crawlers, and AI agents — with UA patterns, verification methods, and recommended access policies.
Not sure what "crawler" vs "scraper" vs "agent" means here? See our terminology guide →
268 results · AI Assistant
A360-Search
ai assistant
Area360
A360-Search
observe
AASA-Bot
ai assistant
Apple
AASA-Bot
observe
adbeat_bot
ai assistant
Adbeat
adbeat_bot
observe
AddThis.com
ai assistant
Oracle
AddThis.com
observe
AdkernelTopicCrawler
ai assistant
AdKernel
AdkernelTopicCrawler
observe
admantx
ai assistant
Integral Ad Science
admantx
observe
AdmixerBot
ai assistant
Admixer
AdmixerBot
observe
AdsBot-Google
ai assistant
AdsBot-Google
observe
AdsBot-Google-Mobile
ai assistant
AdsBot-Google-Mobile
observe
AffsignalCrawler
ai assistant
Affsignal
AffsignalCrawler
observe
AGI Agent
ai assistant
AGI, Inc
observe
signed
AI2Bot-DeepResearchEval
ai assistant
Ai2
AI2Bot-DeepResearchEval
observe
aiHitBot
ai assistant
aiHit
aiHitBot
observe
Amazing-SearchBot
ai assistant
Amazing
Amazing-SearchBot
observe
AmazingTest-SearchBot
ai assistant
Amazing
AmazingTest-SearchBot
observe
Amazon Bedrock AgentCore Browser (AP Northeast)
ai assistant
Amazon
observe
signed
Amazon Bedrock AgentCore Browser (AP South)
ai assistant
Amazon
observe
signed
Amazon Bedrock AgentCore Browser (AP Southeast 1)
ai assistant
Amazon
observe
signed
Amazon Bedrock AgentCore Browser (AP Southeast)
ai assistant
Amazon
observe
signed
Amazon Bedrock AgentCore Browser (EU Central 1)
ai assistant
Amazon
observe
signed
Amazon Bedrock AgentCore Browser (EU West 1)
ai assistant
Amazon
observe
signed
Amazon Bedrock AgentCore Browser (US East 1)
ai assistant
Amazon
observe
signed
Amazon Bedrock AgentCore Browser (US East 2)
ai assistant
Amazon
observe
signed
Amazon Bedrock AgentCore Browser (US West 2)
ai assistant
Amazon
observe
signed
Amazon Buy For Me
ai assistant
Amazon
AmazonBuyForMe
observe
Amazon-Advertising-ad-standards-bot
ai assistant
Amazon
Amazon-Advertising-ad-standards-bot
observe
amazon-QBusiness
ai assistant
Amazon
amazon-QBusiness
observe
AmazonProductDiscovery
ai assistant
Amazon
AmazonProductDiscovery
observe
Amzn-User
ai assistant
Amazon
Amzn-User
observe
AndersPinkBot
ai assistant
Anders Pink
AndersPinkBot
observe
Apify Website Content Crawler
ai assistant
Apify
observe
signed
ArchiveBot
ai assistant
Wikimedia
ArchiveBot
observe
ArenaUnfurlBot
ai assistant
Arena
ArenaUnfurlBot
observe
Assaybot
ai assistant
Index Exchange
Assaybot
observe
AudigentAdBot
ai assistant
Audigent
AudigentAdBot
observe
Automattic
ai assistant
WordPress
Automattic
observe
AwarioBot
ai assistant
Awario
AwarioBot
observe
Bing
ai assistant
Microsoft
Bing
observe
bitlybot
ai assistant
Bitly
bitlybot
observe
BitSightBot
ai assistant
Bitsight
BitSightBot
observe
Blackboard
ai assistant
Anthology
Blackboard
observe
Bloglines
ai assistant
Ask
Bloglines
observe
Bluesky
ai assistant
Bluesky
Bluesky
observe
BomboraBot
ai assistant
Bombora
BomboraBot
observe
BrandVerity
ai assistant
BrandVerity
BrandVerity
observe
Browserbase
ai assistant
Browserbase
Browserbase
observe
signed
BubblesBot
ai assistant
Bubbles
BubblesBot
observe
BublupBot
ai assistant
Bublup
BublupBot
observe
Buck
ai assistant
Hypefactors
Buck
observe
Buttondown
ai assistant
Buttondown
Buttondown
observe
CartAI Agentic Commerce as a Service
ai assistant
CartAI
observe
signed
ChatGPT agent
ai assistant
OpenAI
ChatGPT
observe
signed
ChatGPT-User
ai assistant
OpenAI
ChatGPT-User
observe
CheckMarkNetwork
ai assistant
CheckMark Network
CheckMarkNetwork
observe
Cincraw
ai assistant
CINC
Cincraw
observe
Citoid
ai assistant
Wikimedia
Citoid
observe
Claude-User
ai assistant
Anthropic
Claude-User
observe
Cledara SaaS Management Agent
ai assistant
Cledara
observe
signed
ClickUpLinkUnfurler
ai assistant
ClickUp
ClickUpLinkUnfurler
observe
Cloudflare Browser Run
ai assistant
Cloudflare
Cloudflare
observe
signed