Updated daily
Crawler Catalog
1,876 known bots, crawlers, and AI agents — with UA patterns, verification methods, and recommended access policies.
Not sure what "crawler" vs "scraper" vs "agent" means here? See our terminology guide →
1,876 results
Netcraft
security
Netcraft
Netcraft
allow
NetcraftSurveyAgent
ai assistant
Netcraft
NetcraftSurveyAgent
observe
netEstate
ai scraper
netEstate
netEstate
observe
netEstate Imprint Crawler
ai crawler
netEstate
observe
Neticle
ai assistant
Neticle
Neticle
observe
NetMechanic
other
NetMechanic
observe
NetResearchServer
other
NetResearchServer
observe
NetScope-DomainScanner
other
NetScope-DomainScanner
observe
NetSeer
ai assistant
NetSeer
NetSeer
observe
NetSystemsResearch
other
NetSystemsResearch
observe
Netumo
monitoring
Netumo
netumo
allow
Netvibes
ai assistant
Netvibes
Netvibes
observe
NETVIGIE
monitoring
Netvigie
NETVIGIE
allow
NeuwoBot
ai assistant
Neuwo
NeuwoBot
observe
New
ai coding agent
New Relic
New
observe
New Relic
monitoring
New Relic
allow
New York Times Newsgathering
other
The New York Times
New
observe
NewRelicbot
ai coding agent
New Relic
NewRelicbot
observe
newsai
other
newsai
observe
NewsBank
archiver
NewsBank
allow
NewsBlur
social media
NewsBlur
NewsBlur
allow
NewsBot
other
NewsBot
observe
newsharecounts
other
newsharecounts
observe
Newslitbot
ai assistant
Newslit
Newslitbot
observe
NewsNow
search engine
NewsNowUK
NewsNow
allow
newspaper
seo crawler
newspaper
challenge
NewsRoom.BI
ai assistant
Marfeel
NewsRoom.BI
observe
Newzbin
other
Newzbin
observe
NextCloud
ai assistant
Nextcloud
NextCloud
observe
NextGenSearchBot
other
NextGenSearchBot
observe
Nicecrawler
archiver
NiceCrawler
Nicecrawler
allow
NICErsPRO
other
NICErsPRO
observe
niki-bot
other
niki-bot
observe
Nikto
security
Open-source
challenge
NimbleCrawler
other
NimbleCrawler
observe
Nimbostratus-Bot
other
Nimbostratus-Bot
observe
NINJA
other
NINJA
observe
Nitro-Optimizer-Agent
ai coding agent
NitroPack
Nitro-Optimizer-Agent
observe
Nitro-Warmup-Agent
other
Nitro-Warmup-Agent
observe
NitroBot
seo crawler
NitroPack Ltd
Nitro-
challenge
NIXStatsbot
ai coding agent
NixStats
NIXStatsbot
observe
NixStatsMonitoringBot
monitoring
NixStats
NixStatsMonitoringBot
allow
NLUX_IAHarvester
other
NLUX_IAHarvester
observe
Nmap
security
Nmap
Nmap
allow
NoahBot
other
NoahWire
NoahBot
observe
Nodeping
monitoring
Nodeping
NodePing
allow
Noibu
ai coding agent
Noibu
Noibu
observe
Noibu JS Beautifier
other
Noibu
Noibu
allow
Noorobot
seo crawler
Noor Digital Agency AB
noorobot
challenge
Nooshub
social media
Nooshub
Nooshub
allow
NostoCrawlerBot
seo crawler
Nosto
NostoCrawlerBot
challenge
Nostra
seo crawler
Nostra
challenge
Notabot
search engine
Helpfeel
Notabot
allow
Notion Automation
other
Notion
allow
NovaAct
ai agent
Amazon
NovaAct
observe
Novellum AI Crawl
ai crawler
Novellum
observe
NPBot
other
NPBot
observe
NTENTbot
other
NTENTbot
observe
Nutch
seo crawler
Nutch
challenge
Nuzzel
other
Nuzzel
observe