Updated daily
Crawler Catalog
1,876 known bots, crawlers, and AI agents — with UA patterns, verification methods, and recommended access policies.
Not sure what "crawler" vs "scraper" vs "agent" means here? See our terminology guide →
1,876 results
Dragonbot
seo crawler
Dragon Metrics
Dragonbot
challenge
Dragonfly
ai assistant
Publisher Discovery
Dragonfly
observe
Drata
ai coding agent
Drata
observe
Drata Autopilot
security
Drata
allow
Dratabot
security
Drata
Dratabot
allow
DreamHost
ai coding agent
DreamHost
DreamHost
observe
Drift Agent
ai assistant
Drift Security
observe
signed
drupact
other
drupact
observe
Drupalbot
other
Drupalbot
observe
ds9
ai assistant
Copyright Clearance Center
ds9
observe
Dubbotbot
ai coding agent
DubBot
Dubbotbot
observe
DuckAssistbot
ai assistant
DuckDuckGo
DuckAssistBot
observe
DuckDuckBot
search engine
DuckDuckGo
DuckDuckBot
allow
DuckDuckGo-Favicons-Bot
search engine
DuckDuckGo
DuckDuckGo-Favicons-Bot
allow
DVbot
ai assistant
DoubleVerify
DVbot
observe
e.ventures
other
e.ventures
observe
EasyBib
other
EasyBib
observe
EasyBill Import Manager
other
easybill.de
easybill-ImportManager
observe
EasyCron
other
EasyCron
allow
Easydns
monitoring
EasyDNS
easyDNS
allow
EasyScan
ai assistant
codire GmbH
observe
signed
ec2linkfinder
other
ec2linkfinder
observe
Echobot Bot
ai crawler
Echobox
EchoboxBot
observe
ecoresearchCrawler
other
ecoresearchCrawler
observe
EcoVadisSustainabilityBot
ai assistant
EcoVadis
EcoVadisSustainabilityBot
observe
edisterbot
other
edisterbot
observe
EFF
ai coding agent
Electronic Frontier
EFF
observe
electricmonk
other
electricmonk
observe
Elisabot
other
Elisabot
observe
ellisphere
other
ellisphere
observe
elmah.io
ai coding agent
elmah.io
elmah.io
observe
elmah.io Uptime Monitoring
monitoring
elmah.io
elmahio-uptimebot
allow
EmailCollector
other
EmailCollector
observe
EmailSiphon
other
EmailSiphon
observe
EmailWolf
other
EmailWolf
observe
Embedly
social media
Embedly
Embedly
allow
emetriqContextualBot
other
emetriqContextualBot
observe
eMoney
other
eMoney Advisor
eMoney
observe
eMoney Advisor
seo crawler
eMoney Advisor
eMoneyBot
challenge
epicbot
other
epicbot
observe
Epivoz Crawler
seo crawler
Techmeme
challenge
EpivozCrawler
ai assistant
Techmeme
EpivozCrawler
observe
eRepublik.tools
monitoring
Sebastian Foth - Software Solutions
allow
eright
other
eright
observe
ErisBot
other
ErisBot
observe
ERNIEBot
ai scraper
Baidu
ERNIEBot
observe
EroCrawler
other
EroCrawler
observe
EtaoSpider
other
EtaoSpider
observe
europarchive.org
other
europarchive.org
observe
ev-crawler
ai assistant
Headline
ev-crawler
observe
evc-batch
other
evc-batch
observe
EvernoteRichLinkBot
ai assistant
Evernote
EvernoteRichLinkBot
observe
everyfeed-spider
other
everyfeed-spider
observe
EveryoneSocialBot
other
EveryoneSocialBot
observe
EvoUptimeBot
monitoring
Evo Agency Ltd.
EvoUptimeBot
allow
ExaBot
ai data provider
Exa
ExaBot
observe
Exodus
other
Exodus
allow
Expanse
security
Palo Alto Networks
Expanse
allow
Experibot
other
Experibot
observe
expo-research-bot
other
expo-research-bot
observe