How to block AllAfrica NewsBot
Operated by AllAfrica. AllAfrica NewsBot is a content fetching bot operated by AllAfrica, a pan-African news aggregation service. The bot visits websites to collect and aggregate news content from various sources across the African continent for distribution on their news platform.
nginx
In the server block, then reload.
# In the server block. 403 rather than 444: a closed connection tells the
# operator nothing, and an agent that gets a status code can log it.
if ($http_user_agent ~* "(AllAfrica NewsBot)") {
return 403;
}
Apache
In .htaccess or the vhost.
BrowserMatchNoCase "AllAfrica NewsBot" bad_bot
<RequireAll>
Require all granted
Require not env bad_bot
</RequireAll>
Cloudflare
Security → WAF → Custom rules.
# Security → WAF → Custom rules, action: Block (http.user_agent contains "AllAfrica NewsBot")
WordPress
A child theme's functions.php, or a small plugin.
// functions.php of a child theme, or a small plugin. Runs before WordPress
// builds the page, so a blocked agent costs one PHP process and no queries.
add_action('init', function () {
$agent = $_SERVER['HTTP_USER_AGENT'] ?? '';
foreach (['AllAfrica NewsBot'] as $needle) {
if (stripos($agent, $needle) !== false) {
status_header(403);
exit('Blocked: AllAfrica NewsBot');
}
}
});
A rule on the user agent is a rule on a string it chose
The server rules match on a header the requester sets, which stops the agent that says who it is and not the one that copies somebody else's name. Botscope verifies identity against DNS and published address ranges, records which check decided each request, and shows you the ones that lied.