facebookexternalhit
recommended: observefacebookexternalhit is Meta's web crawler that fetches content from apps or websites shared on Facebook, Instagram, or Messenger to gather, cache, and display information like titles, descriptions, and thumbnail images.
What it does
facebookexternalhit usually makes a single request within seconds of a URL being shared, without crawling further. Traffic rises and falls with sharing activity, so a widely circulated link can produce a sharp burst of fetches.
How to identify it
User-agent contains any of:
facebookexternalhit
robots.txt
Well-behaved crawlers honor robots.txt. To control facebookexternalhit:
Allow full access
User-agent: facebookexternalhit
Allow: /Block entirely
User-agent: facebookexternalhit
Disallow: /
Grey-area scrapers ignore robots.txt. Botscope enforces your policy at the edge or origin regardless —
see how →
See facebookexternalhit on your own site
Botscope shows every bot and AI agent hitting your site — and lets you allow, challenge, or block each one. Observe first, enforce when you're ready.
Start free — connect in minutes