How to Block AI Bots on CodeIgniter 4: Complete 2026 Guide
CodeIgniter 4 calls its request/response interceptors Filters — classes implementing FilterInterface with before() and after() methods. Bot blocking uses before() for the hard 403 (Layers 3–4) and after() for the X-Robots-Tag header (Layer 3), registered globally in app/Config/Filters.php. The robots.txt file (Layer 1) goes in public/ — the document root CodeIgniter points your web server at.
CodeIgniter Filters vs Middleware
ResponseInterface to stop chain. Return null to continue.Protection layers
Layer 1: robots.txt
Place robots.txt in public/ — the directory your web server's document root points to. Apache and nginx serve files from here directly, before CodeIgniter's front controller runs.
# public/robots.txt User-agent: * Allow: / User-agent: GPTBot User-agent: ClaudeBot User-agent: anthropic-ai User-agent: Google-Extended User-agent: CCBot User-agent: Bytespider User-agent: Applebot-Extended User-agent: PerplexityBot User-agent: Diffbot User-agent: cohere-ai User-agent: FacebookBot User-agent: omgili User-agent: omgilibot User-agent: Amazonbot User-agent: DeepSeekBot User-agent: MistralBot User-agent: xAI-Bot User-agent: AI2Bot Disallow: /
Directory layout: public/robots.txt lives alongside public/index.php and public/.htaccess. It is served by your web server before CodeIgniter processes the request — no route needed.
Layer 2: noai meta tag
Add the noai meta tag to your base view layout. Most CodeIgniter apps use a layout file in app/Views/layouts/ or a shared header partial:
<?php /* app/Views/layouts/main.php */ ?>
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="robots" content="<?= esc($robots ?? 'noai, noimageai') ?>">
<title><?= esc($title ?? 'My App') ?></title>
</head>
<body>
<?= $this->renderSection('content') ?>
</body>
</html>Pass a robots variable from your controller to override per-page: return view('page', ['robots' => 'index, follow']). The ?? fallback applies the noai directive globally when no override is passed.
Layers 3 & 4: Filter class
Create a Filter class in app/Filters/. The before() method handles the hard 403 block. The after() method injects the X-Robots-Tag header on all legitimate responses.
app/Filters/AiBotBlocker.php
<?php
namespace App\Filters;
use CodeIgniter\Filters\FilterInterface;
use CodeIgniter\HTTP\RequestInterface;
use CodeIgniter\HTTP\ResponseInterface;
class AiBotBlocker implements FilterInterface
{
private const AI_BOT_PATTERNS = [
'gptbot', 'chatgpt-user', 'oai-searchbot',
'claudebot', 'anthropic-ai', 'claude-web',
'google-extended', 'ccbot', 'bytespider',
'applebot-extended', 'perplexitybot', 'diffbot',
'cohere-ai', 'facebookbot', 'meta-externalagent',
'omgili', 'omgilibot', 'amazonbot',
'deepseekbot', 'mistralbot', 'xai-bot', 'ai2-bot',
];
private const EXEMPT_PATHS = [
'/robots.txt',
'/sitemap.xml',
'/favicon.ico',
];
/**
* Runs before the controller.
* Return a ResponseInterface to short-circuit — no controller runs.
* Return null to continue the filter chain.
*/
public function before(RequestInterface $request, $arguments = null)
{
$path = '/' . ltrim($request->getUri()->getPath(), '/');
// Always pass through exempt paths
if (in_array($path, self::EXEMPT_PATHS, true)) {
return null;
}
$ua = strtolower($request->getHeaderLine('User-Agent'));
foreach (self::AI_BOT_PATTERNS as $pattern) {
if (str_contains($ua, $pattern)) {
// Layer 4: hard 403 block
return service('response')
->setStatusCode(403)
->setContentType('text/plain')
->setBody('Forbidden');
}
}
return null; // Continue to controller
}
/**
* Runs after the controller.
* Only reached by legitimate (non-bot) requests.
* Layer 3: inject X-Robots-Tag on every response.
*/
public function after(RequestInterface $request, ResponseInterface $response, $arguments = null)
{
$response->setHeader('X-Robots-Tag', 'noai, noimageai');
return $response;
}
}Key points
before()returnsnullto continue or aResponseInterfaceto stop. There is no 'next()' function — this is the CodeIgniter convention.service('response')returns the shared response instance. CallingsetStatusCode(403)andsetBody()on it and returning it frombefore()sends the 403 immediately.str_contains()requires PHP 8.0+. For PHP 7.4, usestrpos($ua, $pattern) !== false.$request->getUri()->getPath()returns the path without query string or scheme — safe to use directly for EXEMPT_PATHS comparison.- The
after()method is only reached by legitimate requests — blocked bots are rejected inbefore()and never continue. This means X-Robots-Tag is never set on 403 responses, which is correct.
app/Config/Filters.php
Register the filter globally so it runs on every request:
<?php
namespace Config;
use App\Filters\AiBotBlocker;
use CodeIgniter\Config\Filters as BaseFilters;
class Filters extends BaseFilters
{
/**
* Register filter aliases for use in $globals and $filters.
*/
public array $aliases = [
// ... existing aliases (csrf, honeypot, etc.) ...
'aiBotBlocker' => AiBotBlocker::class,
];
/**
* List of filter aliases that are always applied.
*/
public array $globals = [
'before' => [
// 'honeypot',
// 'csrf',
'aiBotBlocker', // ← add here
],
'after' => [
// 'toolbar',
// Note: X-Robots-Tag is injected inside the filter's after() method
// rather than via $globals['after'] — same effect, self-contained.
],
];
// $methods and $filters can restrict to specific routes — see route-scoped example below
}Route-scoped filtering
To apply the filter only to specific routes (e.g. an API), use the filter option in app/Config/Routes.php. Remove the filter from$globals when using this approach:
<?php
// app/Config/Routes.php
// Single route
$routes->get('products', 'ProductController::index', ['filter' => 'aiBotBlocker']);
// Route group — all routes under /api/ are protected
$routes->group('api', ['filter' => 'aiBotBlocker'], function ($routes) {
$routes->get('products', 'Api\ProductController::index');
$routes->get('orders', 'Api\OrderController::index');
$routes->resource('items', ['controller' => 'Api\ItemController']);
});Route-scoped filtering leaves frontend pages (HTML responses) unblocked and protects only your API routes. Useful if you want X-Robots-Tag only on data endpoints.
Verification
# Layer 1 — robots.txt (must be accessible to all bots) curl https://your-codeigniter-site.com/robots.txt # Layer 3 — X-Robots-Tag on legitimate request curl -I https://your-codeigniter-site.com/products # Expected: X-Robots-Tag: noai, noimageai # Layer 4 — hard block on bot user-agent curl -A "Mozilla/5.0 (compatible; GPTBot/1.0; +https://openai.com/gptbot)" \ -I https://your-codeigniter-site.com/products # Expected: HTTP/1.1 403 Forbidden # robots.txt must remain accessible even to bots curl -A "GPTBot" -I https://your-codeigniter-site.com/robots.txt # Expected: HTTP/1.1 200 OK # Confirm filter is registered: php spark filter:check GET /products # Should show: aiBotBlocker
The php spark filter:check command (CodeIgniter 4.3+) shows which filters are applied to a given route — useful for verifying registration without a browser.
FAQ
What are CodeIgniter Filters and how do they differ from middleware?
Filters are CodeIgniter's equivalent of middleware. A Filter implements FilterInterface with two methods: before() which runs before the controller and after() which runs after. Unlike frameworks where you call a 'next' function, in CodeIgniter you return null to continue or a ResponseInterface to stop. The same class handles both pre and post request processing, making it more self-contained than separate middleware pairs.
Where does robots.txt go in a CodeIgniter 4 project?
Place it in public/ — CodeIgniter's document root. Your web server (Apache or nginx) serves files from this directory directly, before the PHP front controller handles the request. Never place robots.txt in the project root alongside app/ and system/ — your web server does not serve that directory. The full path is public/robots.txt, which is accessible at /robots.txt on your domain.
How do I register the filter globally in CodeIgniter 4?
In app/Config/Filters.php: add your class to the $aliases array (e.g. 'aiBotBlocker' => AiBotBlocker::class), then add the alias string to the $globals['before'] array. Filters in $globals['before'] run on every request before any controller. For route-specific filtering, use the 'filter' option in route definitions instead of $globals.
Why use after() for X-Robots-Tag instead of before()?
The before() method runs before the controller generates a response — there is no ResponseInterface to modify at that point. The after() method receives the complete response object from the controller and can set headers on it before it is sent. Since blocked bots are rejected in before(), they never reach after() — so X-Robots-Tag only appears on responses to legitimate requests. This is exactly the desired behavior.
Does str_contains() work in my CodeIgniter 4 project?
str_contains() was added in PHP 8.0. CodeIgniter 4.4+ requires PHP 8.1+, so str_contains() is safe. If you are on CodeIgniter 4.3 or earlier (which supports PHP 7.4+), replace str_contains($ua, $pattern) with strpos($ua, $pattern) !== false. The logic is identical — strpos returns the character position or false when not found.
Is your site protected from AI bots?
Run a free scan to check your robots.txt, meta tags, and overall AI readiness score.