Updated 16 September 2026
01 · why a page-level script is not enough
Why a page-level script is not enough
A tag that runs in the browser can only see clients that run it. Anything reading your posts with an HTTP library never executes it, which is most automated traffic on a content site.
Classifying at the origin means the decision happens on the request, before the theme, the plugins and the database are involved.
02 · what typically reaches a wordpress site
What typically reaches a WordPress site
| Bot | Operator | Default policy | Verifiable |
|---|---|---|---|
| AI Search | Cloudflare | Observe | By user agent |
| AIWebIndex-Agent | Lyrenth | Observe | By user agent |
| Amazon Kendra | Amazon | Observe | By user agent |
| Amazonbot | Amazon | Observe | By user agent |
| Amzn-SearchBot | Amazon | Observe | By user agent |
| Anchor Browser | Anchor | Observe | Signed requests |
| Anomura | Direqt | Observe | By user agent |
| Applebot | Apple | Observe | By user agent |
| Atlassian (Rovo AI) | Atlassian | Allow | By user agent |
| atlassian-bot | Atlassian | Observe | By user agent |
Read live from the Botscope crawler catalogue. Default policy is the catalogue default; every agent can be overridden per site.
03 · getting it running
Getting it running
- 1 Install the Botscope plugin from wp-admin — Plugins, Add New, Upload. It carries the origin agent and puts it in place for you, so there is no file to upload over FTP and no php.ini to edit.
- 2 Paste the site ID, API key and secret into Botscope, Settings. Nothing is classified until they are there.
- 3 Leave the site in observe mode and let a week of traffic accumulate.
- 4 Review which agents arrived and set a policy for each.
- 5 Generate robots.txt from that same policy so the declared and enforced rules agree.
04 · specific to this platform
Specific to this platform
The plugin has to get the agent loading before WordPress does, and how it can do that depends on the host: a PHP-FPM host reads a .user.ini, a mod_php host reads .htaccess, and a few managed hosts permit neither. The plugin writes both, tries three load positions in order, and tells you on its own settings screen which one it actually got — so a host that will not allow it says so rather than looking installed and classifying nothing.
Where none of the positions is available, the plugin still runs inside WordPress. That is later than the origin agent and it misses anything served before WordPress boots, which is a real reduction in coverage and is worth knowing before choosing this over DNS.
xmlrpc.php and the REST API are frequent automation targets and are worth their own policy rather than inheriting the site default.
Questions
Does this slow down page loads?
The classification happens on the request path and is designed to add negligible latency. Verification lookups that would be slow are cached after the first request from an address.
Does it work with caching plugins?
Yes. The agent sees the request before the cache layer answers it, so cached responses are still classified.
Will it interfere with Jetpack or my SEO plugin?
No. Those are allowed agents in the catalogue and stay allowed by default.
What if I am on managed hosting?
The plugin reports which of the three load positions your host allowed, on its own settings screen. Where none is available it still runs inside WordPress, which is later and sees less; the DNS method does not depend on the host at all.
Do I need FTP or SSH?
No. The plugin is a zip uploaded through wp-admin, and the three credentials are pasted into its settings screen. It carries the agent and places it itself.
Is the plugin on wordpress.org?
Not yet. Until it is, the zip is downloaded from your Botscope panel on the WordPress tab of the connect screen.
See which of these reach your site
Free plan, no card: 100,000 events a month with every feature on. Nothing is blocked until you turn enforcement on.