Updated daily
Crawler Catalog
1,876 known bots, crawlers, and AI agents — with UA patterns, verification methods, and recommended access policies.
Not sure what "crawler" vs "scraper" vs "agent" means here? See our terminology guide →
812 results · Other
SitesOverPagesBot
other
SitesOverPagesBot
observe
SiteSucker
other
SiteSucker
observe
SiteUptime.com
other
SiteUptime.com
observe
Skroutz ImageBot
other
Skroutz S.A.
observe
SkyWatch
other
SkyWatch
observe
SlaccaleBot
other
SlaccaleBot
observe
Slackbot
other
Slack
Slackbot
allow
SleepBot
other
SleepBot
observe
Sleuth
other
Sleuth
observe
slurp
other
slurp
observe
Smallwebindexbot
other
Smallwebindexbot
observe
SmartBotCrawler
other
SmartBotCrawler
observe
Snipcart
other
Snipcart
allow
SocialBeeAgent
other
SocialBeeAgent
observe
SocialRankIOBot
other
SocialRankIOBot
observe
Sonic
other
Sonic
observe
Sora Caisse POS
other
Sora Caisse POS
observe
Sosospider
other
Sosospider
observe
SpankBot
other
SpankBot
observe
spanner
other
spanner
observe
Spark Shipping
other
Spark Shipping
allow
SparkpostBot
other
Bird
allow
spbot
other
spbot
observe
SpeedyIndexBot
other
SpeedyIndexBot
observe
SpiderLing
other
SpiderLing
observe
Spinn3r
other
Spinn3r
observe
spotter
other
spotter
observe
SpringserveBot
other
SpringserveBot
observe
SputnikBot
other
SputnikBot
observe
SQWatcher
other
SQWatcher
observe
star-finder.de
other
star-finder.de
observe
StatistikAustria
other
Statistik Bot
StatistikAustria
observe
StorygizeBot
other
StorygizeBot
observe
Streamline3Bot
other
Streamline3Bot
observe
Stripe
other
Stripe
allow
stripe_apple_pay_verify_ownership
other
Stripe
allow
Stripebot
other
Stripe
Stripebot
allow
SubspaceBot
other
SubspaceBot
observe
SumbleBot
other
SumbleBot
observe
SummalyBot
other
SummalyBot
observe
Summerflare
other
Summerflare
observe
Summify
other
Summify
observe
Supabase
other
Supabase
observe
SuperBot
other
SuperBot
observe
superpagesbot2.0
other
superpagesbot2.0
observe
SurliBot
other
SurliBot
observe
SurveyBot
other
SurveyBot
observe
suzuran
other
suzuran
observe
sv-watchagent
other
sv-watchagent
observe
Svix Webhooks
other
Svix Inc.
allow
Swiftbot
other
Swiftbot
observe
SWIMGBot
other
SWIMGBot
observe
SynthesiBot
other
SynthesiBot
observe
Synthesio
other
Synthesio
observe
Sysomos
other
Sysomos
observe
Szukacz
other
Szukacz
observe
tagoobot
other
tagoobot
observe
Talkwater
other
Talkwater
observe
TangibleeBot
other
TangibleeBot
observe
TaraGroup
other
TaraGroup
observe