Updated daily
Crawler Catalog
1,953 known bots, crawlers, and AI agents — with UA patterns, verification methods, and recommended access policies.
Not sure what "crawler" vs "scraper" vs "agent" means here? See our terminology guide →
The policy each agent gets before a site sets its own. Click a number to list those agents.
1,284 results
Omgilibot
other
Omgilibot
observe
OnlineOrNot.com_bot
ai coding agent
OnlineOrNot
OnlineOrNot.com_bot
observe
Openbot
other
Openbot
observe
opencode
ai coding agent
Anomaly
opencode
observe
openfind
other
openfind
observe
Openfind data gatherer
other
observe
OpenGraph.io
ai assistant
Opengraph
OpenGraph.io
observe
OpenGraphCheck
other
OpenGraphCheck
observe
OpenHoseBot
other
OpenHoseBot
observe
OpenTheBoxBot
other
OpenTheBoxBot
observe
opinion-tracker
other
opinion-tracker
observe
Oracle Ultra Search
other
Oracle
observe
OrangeBot
other
OrangeBot
observe
Orbbot
other
Orbbot
observe
Orlo-LinkPreview
ai assistant
Orlo
Orlo-LinkPreview
observe
Orthogaffe
other
Orthogaffe
observe
outboundbuddy-enrichment
ai assistant
Outbound Buddy
outboundbuddy-enrichment
observe
OutclicksBot
other
OutclicksBot
observe
Owler
ai assistant
Owler
Owler
observe
OWLer-W
other
OWLer-W
observe
page-preview-tool
ai assistant
SalesViewer
page-preview-tool
observe
Page2RSS
other
Page2RSS
observe
PagePeeker
ai assistant
PagePeeker
PagePeeker
observe
PageThing
other
PageThing
observe
Pandalytics
ai assistant
Domainsbot
Pandalytics
observe
PanguBot
ai scraper
Huawei
PanguBot
observe
panscient.com
ai assistant
Panscient
panscient.com
observe
Paqlebot
other
Paqle
Paqlebot
observe
parisbot
other
parisbot
observe
parsely.com
ai coding agent
Parse.ly
parsely.com
observe
PartnerOptimizer-bot
ai assistant
PartnerOptimizer
PartnerOptimizer-bot
observe
Payhawk Invoice Fetching Agent
ai assistant
Payhawk
payhawk-invoice-fetching-bot
observe
signed
payroll-bot
ai crawler
ADP
observe
PDFBot
other
PDFBot
observe
peer39_crawler
other
Peer39
peer39_crawler
observe
PerMan
other
PerMan
observe
Perplexity-User
ai assistant
Perplexity
Perplexity-User
observe
Perplexity‑User
ai assistant
Perplexity
Perplexity-User
observe
signed
PerplexityBot
ai crawler
Perplexity
PerplexityBot
observe
PetalBot
ai crawler
Huawei
PetalBot
observe
PhindBot
ai assistant
Phind
PhindBot
observe
PiBot
other
PiBot
observe
Pingdom.com_bot
ai coding agent
SolarWinds
Pingdom.com_bot
observe
pingping.io
ai coding agent
PingPing
pingping.io
observe
PiplBot
other
PiplBot
observe
Piqo Podsync
ai assistant
Piqo
Piqo
observe
Pixalate.com
other
Pixalate.com
observe
PlagAwareBot
other
PlagAwareBot
observe
Plesk screenshot bot
other
Plesk
observe
PocketParser
ai assistant
PocketParser
observe
PodchaserParser
ai assistant
Podchaser
PodchaserParser
observe
Podimo
ai assistant
Podimo
Podimo
observe
Poduptime
other
Poduptime
observe
Poggio-Citations
ai assistant
Poggio
Poggio-Citations
observe
Pomothy-Bot
other
Pomothy-Bot
observe
Poseidon Research Crawler
other
observe
postrank
other
postrank
observe
PR-CY.RU
other
PR-CY.RU
observe
PressEngineBot
ai assistant
PressEngine
PressEngineBot
observe
PriEcoBot
other
PriEcoBot
observe