Updated daily
Crawler Catalog
1,876 known bots, crawlers, and AI agents — with UA patterns, verification methods, and recommended access policies.
Not sure what "crawler" vs "scraper" vs "agent" means here? See our terminology guide →
1,876 results
Bibliotheque
other
Bibliothèque nationale de France
Bibliotheque
observe
Bibliotheque Nacional de France Crawler
ai scraper
Bibliothèque nationale de France
observe
bidswitchbot
other
bidswitchbot
observe
Big Sur AI
ai crawler
Big Sur AI
bigsur.ai
observe
BIGLOTRON
other
BIGLOTRON
observe
BigScootsMonitor
monitoring
BigScoots
allow
BigUpData Bot
seo crawler
BigUpData
BigUpDataBot
challenge
BinaryCanary
monitoring
Binary Canary
binarycanary
allow
Bing
ai assistant
Microsoft
Bing
observe
Bing Ads
seo crawler
Microsoft
adidxbot
challenge
Bing Preview
social media
Microsoft
BingPreview
allow
BingBot
search engine
Microsoft
bingbot
allow
binlar
other
binlar
observe
Birdcrawlerbot
other
Birdcrawlerbot
observe
BitBot
other
BitBot
observe
Bitbucket
other
Atlassian
allow
bitlybot
ai assistant
Bitly
bitlybot
observe
BitSightBot
ai assistant
Bitsight
BitSightBot
observe
bl.uk_lddc_bot
archiver
British Library
bl.uk_lddc_bot
allow
Black
other
Black
observe
Black Duck Fast Dynamic
security
Black Duck
Black
allow
Blackboard
ai assistant
Anthology
Blackboard
observe
BlackVeil-Security-Scanner
security
BlackVeil
BlackVeil-Security-Scanner
allow
Blekkobot
other
Blekkobot
observe
BLEXBot
seo crawler
WebMeUp
BLEXBot
challenge
Bling ERP
other
Bling
Bling
observe
Blockaid
security
Blockaid
Blockaid
allow
Bloglines
ai assistant
Ask
Bloglines
observe
blogmuraBot
other
blogmuraBot
observe
Blogtrottr
social media
Blogtrottr
Blogtrottr
allow
BlogVault
monitoring
BlogVault
BlogVault
allow
BlowFish
other
BlowFish
observe
BLP_bbot
other
BLP_bbot
observe
Bluesky
ai assistant
Bluesky
Bluesky
observe
Bluesky Link Preview Service
social media
Bluesky
allow
bne.es_bot
archiver
bne.es_bot
allow
bnf.fr_bot
archiver
Bibliothèque nationale de France
bnf.fr_bot
allow
BoardGamePrices Bot
seo crawler
KP Software Consult ApS
BoardGamePrices
challenge
BomboraBot
ai assistant
Bombora
BomboraBot
observe
BOMForge-DeepCrawler
other
BOMForge-DeepCrawler
observe
Bookmark
other
Bookmark
observe
BorderxBot
ai crawler
Borderxlab (Nubestore AI)
observe
bot-pge.chlooe.com
other
bot-pge.chlooe.com
observe
Bot.AraTurka.com
other
Bot.AraTurka.com
observe
BotALot
other
BotALot
observe
Botify
seo crawler
Botify
botify
challenge
BotRightHere
other
BotRightHere
observe
BoxcarBot
other
BoxcarBot
observe
br-crawler
other
br-crawler
observe
brainobot
other
brainobot
observe
BrandONbot
other
BrandONbot
observe
BrandVerity
ai assistant
BrandVerity
BrandVerity
observe
Brandwatch
ai crawler
Brandwatch
observe
Bravebot
ai crawler
Brave Software, Inc.
Bravebot
observe
Brightbot
ai data provider
Bright Data
Brightbot
observe
BrightEdge Bot
seo crawler
BrightEdge
BrightEdge
challenge
brokenlinkcheck.com
other
brokenlinkcheck.com
observe
BrokenLinksBot
other
BrokenLinksBot
observe
Browserbase
ai assistant
Browserbase
Browserbase
observe
signed
Brynson Compliance Bot
monitoring
Brynson
allow
signed