Crawler Catalog
1,953 known bots, crawlers, and AI agents — with UA patterns, verification methods, and recommended access policies.
Not sure what "crawler" vs "scraper" vs "agent" means here? See our terminology guide →
The policy each agent gets before a site sets its own. Click a number to list those agents.
1,953 results
OneTrust
OneTrust
OneTrust, LLC.
Onetrust
Online Webceo
online-webceo-bot
OnlineOrNot
OnlineOrNot.com_bot_1.0_(https:
OnlineOrNot
OnlineOrNot.com_bot
Openbot
Anomaly
opencode
openfind
Opengraph
Opengraph
OpenGraph.io
OpenGraphCheck
OpenHoseBot
Openindex
OpenindexSpider
OpenRSS
OpenRSS
OpenTheBoxBot
opinion-tracker
Oracle
OrangeBot
Orbbot
Orlo
Orlo-LinkPreview
Orlo
Orlo-LinkPreview
Orthogaffe
OurFamilyWizard
Cozi-iCalendar-FeedReader
Outbound Buddy
outboundbuddy-enrichment
Outbrain
outbrain
OutclicksBot
Outsell Corporation
OutsellURLValidator
Overcast Radio
Overcast
Owler
Owler
OWLer-W
Ozon
Ozon
Paddle
Paddle
Paessler
PRTGCloudBot
SalesViewer
page-preview-tool
Page2RSS
PagePeeker
PagePeeker
PageThing
Domainsbot
Pandalytics
Huawei
PanguBot
Panopta
Panopta
Panscient
panscient.com
Paqle
Paqlebot
parisbot
Parse.ly
parse.ly scraper
Parse.ly
parsely.com
particlenewsbot
PartnerOptimizer
PartnerOptimizer-bot
Payhawk
payhawk-invoice-fetching-bot
PayPal
PayPal
ADP
PDFBot
Peer39
peer39_crawler
PerMan
Perplexity
Perplexity-User
Perplexity
Perplexity-User
Perplexity
PerplexityBot
Huawei
PetalBot
Phind
PhindBot
PiBot