Botscope
What people bring to us
Each of these starts with traffic you cannot account for. Pick the one that sounds like your week.
- 1,953
- bots and agents catalogued
- 774
- named operators
- 14
- categories
AI agents
-
llms.txt, and what it does not do
llms.txt is a proposal, not a standard, and nothing obliges an AI agent to read it. What the file is, how to write one, and how to find out who honoured it.
Read → -
Blocking AI scrapers
Not every AI crawler wants the same thing. Separate training collection from assistant fetches and search indexing, then set a policy for each one deliberately.
Read → -
AI coding agents on your docs
Coding assistants fetch documentation on a developer’s behalf, often at machine pace. See which ones reach your docs and decide what each is allowed to read.
Read → -
Enforcing robots.txt
robots.txt asks politely and the operators who ignore it are the ones you wrote it for. How to enforce the same policy at the origin and keep the two in step.
Read →
Measurement
-
Bot traffic in GA4
GA4 filters known spiders by user agent, the one signal a scraper controls. How to see which automated clients actually reach your site and what each is for.
Read → -
Traffic from AI answers
Visits from AI assistants often land in analytics as direct traffic. How to attribute them and connect the crawl that fed the answer to the click it produced.
Read → -
Invalid traffic detection
Find out what share of your reported traffic is not human. Per-request classification gives you a number you can defend to an advertiser or a board.
Read →
Protection
-
Fake Googlebot detection
Anyone can send Googlebot’s user agent, so allowlisting that string hands attackers a bypass. How reverse DNS and published ranges actually verify a crawler.
Read → -
Content scraping
Scraping cannot be made impossible, only expensive. What actually raises the cost, what only moves the problem, and how to tell which is happening on your site.
Read → -
Bot detection software that shows its work
Botscope classifies every request as human, search crawler, AI agent, or scraper and shows the evidence behind each decision. No opaque risk scores.
Read → -
Headless browser detection
Puppeteer, Playwright and Selenium do not announce themselves. How an automated browser is told apart from a real one, and what each check can prove.
Read → -
Credential stuffing detection
Credential stuffing arrives as ordinary login attempts. Which request signals separate a stuffing run from a person mistyping their own password.
Read → -
Fake signup detection
Spam registrations pass validation because they are real submissions. How the request behind the form is classified rather than the data inside it.
Read → -
Proxy and datacenter traffic detection
A residential proxy makes automated traffic look domestic. What an address can and cannot prove, and which checks survive a rotating proxy pool.
Read →