Crawler Catalog
1,876 known bots, crawlers, and AI agents — with UA patterns, verification methods, and recommended access policies.
Not sure what "crawler" vs "scraper" vs "agent" means here? See our terminology guide →
122 results · Monitoring
360Monitoring
360monitoring
AccessStatus
AccessStatus
ActiveComply
ActiveComply
Adagio
Adagiobot
Advailo
AgencyToolsPro
Smartbear
AlertSite
Area360
Audisto
Audisto
Automaton
Automaton
Better Stack
Better
BigScoots
Binary Canary
binarycanary
BlogVault
BlogVault
Brynson
Bugs Nag
BugsNag
BuiltWith
Catchpoint
Catchpoint
Checkly
Checkly
CheckView.io
Cloudflare
Cloudflare
CloudflareDiagnostics
Cloudflare
Cloudtrellis
Cloudtrellis
Cludo
Cludo
ContentKing
contentking
Contentsquare
CrazyEgg
Crazy
DLC Websites
deadlinkchecker
Lumar
deepcrawl
Dotcom-Monitor
Dr. Link Check
EasyDNS
easyDNS
elmah.io
elmahio-uptimebot
Sebastian Foth - Software Solutions
Evo Agency Ltd.
EvoUptimeBot
FTW Entertainment LLC
FDL
Foregenix Limited
Full Story
FullStoryBot
Gatus
Ghost Inspector
Ghost
Grafana Labs
Grafana Labs
GTmetrix
GTmetrix
Guest Posts
GuestpostsBot
HetrixTools Inc
HetrixTools
Honeybadger Industries
HoneybadgerBot
HostTracker
HostTracker
Hotjar
Hotjar
Internet Archive
InternetArchiveBot
IsDownBot
IsDownBot
Jagged Pixel Inc.
Automattic
Jetpack
Kargo
KargoBot-Artemis
Kinsta
kinsta-bot
LinkTiger, Inc.
LinkTiger
LogicMonitor
logicmonitor
Macrobondbot
Macrobondbot
Mavifinds
Mavifinds