Updated daily
Crawler Catalog
1,876 known bots, crawlers, and AI agents — with UA patterns, verification methods, and recommended access policies.
Not sure what "crawler" vs "scraper" vs "agent" means here? See our terminology guide →
74 results · Security
Splunk Attack Analyzer
security
Splunk
allow
sqlmap
security
Open-source
challenge
SSL
security
Qualys
SSL
allow
Sucuri
security
Sucuri
Sucuri
allow
TwinWaveScanner
security
Splunk
TwinWaveScanner
allow
Usercentricsbot
security
Usercentrics
Usercentricsbot
allow
Verispider
security
Project Honey Pot
Verispider
allow
VirusTotalScan
security
VirusTotal
allow
visionheight.com/scan
security
visionheight.com/scan
allow
WebTotemBot
security
WebTotem
allow
Wordfence Central
security
Wordfence
allow
WPScan
security
WPScan LLC
challenge
WTotem
security
WebTotem
WTotem
allow
WTotemBot
security
WebTotem
WTotemBot
allow