Updated daily
Crawler Catalog
1,953 known bots, crawlers, and AI agents — with UA patterns, verification methods, and recommended access policies.
Not sure what "crawler" vs "scraper" vs "agent" means here? See our terminology guide →
The policy each agent gets before a site sets its own. Click a number to list those agents.
1,284 results
UsineNouvelleCrawler
other
UsineNouvelleCrawler
observe
UT-Dorkbot
other
UT-Dorkbot
observe
Validator.nu
ai coding agent
W3C
Validator.nu
observe
VCI
other
VCI
observe
VCI WebViewer VCI WebViewer Win32
other
observe
vebidoobot
other
vebidoobot
observe
vecteurplus
other
vecteurplus
observe
VelenPublicWebCrawler
ai scraper
Velen
VelenPublicWebCrawler
observe
Ventus Agent
ai assistant
ventus-agent
observe
signed
Veoozbot
other
Veoozbot
observe
Verity
ai assistant
GumGum
Verity
observe
verticalsearch
other
verticalsearch
observe
Viber
ai assistant
Viber
Viber
observe
Vigil
other
Vigil
observe
virustotal
ai assistant
VirusTotal
virustotal
observe
Visually.io Shopify Editor
ai assistant
Visually.io
visually-io-shopify-editor
observe
signed
VKRobot
other
VKRobot
observe
vkShare
ai assistant
VK
vkShare
observe
vmcrawl
other
vmcrawl
observe
VoilaBot
other
VoilaBot
observe
voltron
other
voltron
observe
VoluumDSP-content-bot
other
VoluumDSP-content-bot
observe
vsw
other
vsw
observe
vuhuvBot
other
vuhuvBot
observe
W3C_CSS_Validator
ai coding agent
W3C
W3C_CSS_Validator
observe
W3C_I18n-Checker
other
W3C_I18n-Checker
observe
W3C_Unicorn
other
W3C_Unicorn
observe
W3C_Validator
ai coding agent
W3C
W3C_Validator
observe
W3C-checklink
ai coding agent
W3C
W3C-checklink
observe
W3C-mobileOK
other
W3C-mobileOK
observe
WadooBot
other
WadooBot
observe
WanScannerBot
other
WanScannerBot
observe
WARDBot
ai crawler
WEBSPARK
WARDBot
observe
WASALive-Bot
other
WASALive-Bot
observe
WAVE Streaming Platform
ai assistant
wave-streaming-platform
observe
signed
Wayflyer
other
Wayflyer
Wayflyer
observe
wbsearchbot
other
wbsearchbot
observe
Web Image Collector
other
observe
web-archive-net.com.bot
other
web-archive-net.com.bot
observe
WebAuto
other
WebAuto
observe
WebBandit
other
WebBandit
observe
WebCapture 2.0
other
WebCapture
observe
webcompanycrawler
other
webcompanycrawler
observe
WebCopier
other
WebCopier
observe
WebCopier v.2.2
other
observe
WebCopier v3.2a
other
observe
WebCrawler
other
WebCrawler
observe
WebDataStats
other
WebDataStats
observe
WebEnhancer
other
WebEnhancer
observe
WebmasterWorldForumBot
other
WebmasterWorldForumBot
observe
webmon
other
webmon
observe
weborama-fetcher
ai assistant
Weborama
weborama-fetcher
observe
WebReaper
other
WebReaper
observe
WebSauger
other
WebSauger
observe
WebScraperBot
other
WebScraperBot
observe
websearch-bot
other
websearch-bot
observe
Website Quester
other
Website
observe
Website-info.net-Robot
other
Website-info.net-Robot
observe
WebsiteOps
ai coding agent
Watchful
WebsiteOps
observe
websitepulse checker
ai coding agent
WebSitePulse
websitepulse checker
observe