NewsBlur Page Fetcher
recommended: observeNewsBlur Page Fetcher is a bot operated by NewsBlur, an RSS feed reader and news aggregation service. It fetches content from websites on behalf of subscribers, with the user agent number indicating how many users are subscribed to that feed.
What it does
NewsBlur Page Fetcher usually makes a single request within seconds of a URL being shared, without crawling further. Traffic rises and falls with sharing activity, so a widely circulated link can produce a sharp burst of fetches.
How to identify it
User-agent contains any of:
NewsBlur Page Fetcher
robots.txt
Well-behaved crawlers honor robots.txt. To control NewsBlur Page Fetcher:
Allow full access
User-agent: NewsBlur Page Fetcher
Allow: /Block entirely
User-agent: NewsBlur Page Fetcher
Disallow: /
Grey-area scrapers ignore robots.txt. Botscope enforces your policy at the edge or origin regardless —
see how →
See NewsBlur Page Fetcher on your own site
Botscope shows every bot and AI agent hitting your site — and lets you allow, challenge, or block each one. Observe first, enforce when you're ready.
Start free — connect in minutes