Skip to content
Eleventy · 11ty · Static Site Generator·9 min read

How to Block AI Bots on Eleventy (11ty): Complete 2026 Guide

Eleventy builds a static site — there is no server process to intercept requests at runtime. Protecting your content requires two things: getting robots.txt into _site/ (not as obvious as it sounds) and adding a blocking layer in front of the static output. This guide covers both, plus noai meta tags in every major Eleventy template language and X-Robots-Tag headers via Netlify, Vercel, Cloudflare Pages, and nginx.

Eleventy 2.x and 3.x

Examples use both versions. Eleventy 3.x is ESM-first: the config file is eleventy.config.js with export default. Eleventy 2.x uses .eleventy.js with module.exports. The addPassthroughCopy() API is identical in both.

Methods at a glance

MethodWhat it doesBlocks JS-less bots?
addPassthroughCopy → robots.txtSignals crawlers to stay outSignal only
noai meta in base layoutOpt out of AI training site-wide✓ (static HTML)
Front matter robots variablePer-page noai control✓ (static HTML)
X-Robots-Tag (netlify.toml)noai header on all responses✓ (header)
Netlify Edge FunctionHard 403 before static file serves✓
Vercel Edge MiddlewareHard 403 before static file serves✓
Cloudflare Pages FunctionHard 403 at edge globally✓
nginx map blockHard 403 at reverse proxy layer✓

1. robots.txt — addPassthroughCopy

Eleventy only copies files it recognises as templates or that you explicitly declare as passthrough. Without addPassthroughCopy(), your robots.txt is silently dropped from _site/ on every build. This is the most common Eleventy SEO mistake.

Input directory gotcha

The path passed to addPassthroughCopy() is relative to the project root, not the input directory. If your input directory is src, place robots.txt in src/robots.txt and call addPassthroughCopy("src/robots.txt"). Using "robots.txt" without the prefix will silently fail when an input directory is configured.

robots.txt

User-agent: GPTBot
Disallow: /

User-agent: ChatGPT-User
Disallow: /

User-agent: OAI-SearchBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Claude-Web
Disallow: /

User-agent: anthropic-ai
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: Bytespider
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: PerplexityBot
Disallow: /

User-agent: Applebot-Extended
Disallow: /

User-agent: *
Allow: /

eleventy.config.js — Eleventy 3.x (ESM)

// eleventy.config.js (Eleventy 3.x — ESM)
export default function (eleventyConfig) {
  // Project root → robots.txt (no input directory configured)
  eleventyConfig.addPassthroughCopy("robots.txt");

  // If input directory is "src":
  // eleventyConfig.addPassthroughCopy("src/robots.txt");

  // Copy entire public/ folder (covers favicon, images, etc.)
  // eleventyConfig.addPassthroughCopy("public");

  return {
    dir: {
      input: "src",   // remove if using project root as input
      output: "_site",
    },
  };
}

.eleventy.js — Eleventy 2.x (CommonJS)

// .eleventy.js (Eleventy 2.x — CommonJS)
module.exports = function (eleventyConfig) {
  // Same API — just different module syntax
  eleventyConfig.addPassthroughCopy("robots.txt");
  // or eleventyConfig.addPassthroughCopy("src/robots.txt");

  return {
    dir: {
      input: "src",
      output: "_site",
    },
  };
};

Verify after build: ls _site/robots.txt — if missing, the path passed to addPassthroughCopy() is wrong.

2. noai meta tag in layouts

Eleventy renders templates to static HTML at build time — the noai meta tag is in every response before JavaScript runs. Add it to your base layout so every page inherits it. Use front matter for per-page control.

Nunjucks base layout (_includes/base.njk)

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />

  {# Global opt-out — override per page via front matter #}
  <meta name="robots" content="{{ robots | default('noai, noimageai') }}" />

  <title>{{ title }}</title>
</head>
<body>
  {{ content | safe }}
</body>
</html>

Pages that need different behaviour set robots in front matter. The | default() filter uses 'noai, noimageai' site-wide unless overridden.

Page front matter override

---
title: Press Kit
layout: base.njk
# This page can be indexed by search but not used for AI training
robots: "noai, noimageai"
---

# This page opts out of AI training globally.

---
title: Public Blog Post
layout: base.njk
# Explicitly allow everything (override the global noai default)
robots: "index, follow"
---

# This post is freely indexable.

Liquid base layout (_includes/base.liquid)

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />

  {%- assign meta_robots = robots | default: "noai, noimageai" -%}
  <meta name="robots" content="{{ meta_robots }}" />

  <title>{{ title }}</title>
</head>
<body>
  {{ content }}
</body>
</html>

WebC base layout (_includes/base.webc)

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <!-- WebC: use @text to output the variable value -->
  <meta name="robots" :content="robots || 'noai, noimageai'" />
  <title @text="title"></title>
</head>
<body @html="content"></body>
</html>

HTML template (_includes/base.html)

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <!-- HTML templates use Nunjucks syntax by default in Eleventy -->
  <meta name="robots" content="{{ robots | default('noai, noimageai') }}" />
  <title>{{ title }}</title>
</head>
<body>
  {{ content | safe }}
</body>
</html>

Global data file approach

Set a site-wide default in _data/site.json so you do not repeat the value in every template:

// _data/site.json
{
  "robots": "noai, noimageai"
}

// In any Nunjucks template:
// <meta name="robots" content="{{ site.robots }}" />

3. X-Robots-Tag response headers

X-Robots-Tag: noai, noimageai works even when bots ignore the meta tag. Set it via your hosting platform — Eleventy has no server process to set headers itself.

Netlify — netlify.toml

# netlify.toml
[build]
  publish = "_site"
  command = "npx @11ty/eleventy"

[[headers]]
  for = "/*"
  [headers.values]
    X-Robots-Tag = "noai, noimageai"
    X-Content-Type-Options = "nosniff"

Netlify — _headers file

# Place in _site/_headers (or configure passthrough from src/_headers)
# Netlify reads this from the publish directory automatically.
/*
  X-Robots-Tag: noai, noimageai

Vercel — vercel.json

{
  "buildCommand": "npx @11ty/eleventy",
  "outputDirectory": "_site",
  "headers": [
    {
      "source": "/(.*)",
      "headers": [
        { "key": "X-Robots-Tag", "value": "noai, noimageai" }
      ]
    }
  ]
}

Cloudflare Pages — _headers

# _site/_headers (pass through from source via addPassthroughCopy)
/*
  X-Robots-Tag: noai, noimageai

4. Hard 403 blocking

robots.txt and meta tags rely on bot compliance. For a hard block, add a layer that runs before the static file is served. All of the following intercept the request at the edge or proxy before Eleventy's output is touched.

Netlify Edge Function

// netlify/edge-functions/block-ai-bots.ts
import type { Config, Context } from "@netlify/edge-functions";

const AI_BOT_UA = /GPTBot|ChatGPT-User|OAI-SearchBot|ClaudeBot|Claude-Web|anthropic-ai|Google-Extended|Bytespider|CCBot|PerplexityBot|Applebot-Extended|DuckAssistBot|cohere-ai|Meta-ExternalAgent|Diffbot|YouBot|Amazonbot/i;

export default async function (request: Request, context: Context) {
  const url = new URL(request.url);

  // Always allow robots.txt
  if (url.pathname === "/robots.txt") {
    return context.next();
  }

  const ua = request.headers.get("user-agent") ?? "";
  if (AI_BOT_UA.test(ua)) {
    return new Response("Forbidden", { status: 403 });
  }

  return context.next();
}

export const config: Config = {
  path: "/*",
};

// netlify.toml — declare the edge function
// [[edge_functions]]
//   path = "/*"
//   function = "block-ai-bots"

Vercel Edge Middleware (middleware.js)

// middleware.js — place at project root (next to vercel.json)
// Works with Vercel's static output; runs before static files are served.
import { NextResponse } from "next/server";

const AI_BOT_UA = /GPTBot|ChatGPT-User|OAI-SearchBot|ClaudeBot|Claude-Web|anthropic-ai|Google-Extended|Bytespider|CCBot|PerplexityBot|Applebot-Extended|DuckAssistBot|cohere-ai|Meta-ExternalAgent|Diffbot|YouBot|Amazonbot/i;

export function middleware(request) {
  if (request.nextUrl.pathname === "/robots.txt") {
    return NextResponse.next();
  }

  const ua = request.headers.get("user-agent") ?? "";
  if (AI_BOT_UA.test(ua)) {
    return new Response("Forbidden", { status: 403 });
  }

  return NextResponse.next();
}

export const config = {
  matcher: ["/((?!_next/static|_next/image|favicon.ico).*)"],
};

// Note: Vercel Edge Middleware requires @vercel/edge or next.
// For a pure static Eleventy site, the Netlify Edge Function above
// or Cloudflare Pages approach may be simpler.

Cloudflare Pages Functions (_middleware.ts)

// functions/_middleware.ts
// Place in _site/functions/ or configure passthrough via addPassthroughCopy

import type { PagesFunction } from "@cloudflare/workers-types";

const AI_BOT_UA = /GPTBot|ChatGPT-User|OAI-SearchBot|ClaudeBot|Claude-Web|anthropic-ai|Google-Extended|Bytespider|CCBot|PerplexityBot|Applebot-Extended|DuckAssistBot|cohere-ai|Meta-ExternalAgent|Diffbot|YouBot|Amazonbot/i;

export const onRequest: PagesFunction = async ({ request, next }) => {
  const url = new URL(request.url);

  if (url.pathname === "/robots.txt") {
    return next();
  }

  const ua = request.headers.get("user-agent") ?? "";
  if (AI_BOT_UA.test(ua)) {
    return new Response("Forbidden", { status: 403 });
  }

  return next();
};

nginx — self-hosted

# /etc/nginx/conf.d/eleventy-site.conf

map $http_user_agent $blocked_bot {
    default                 0;
    "~*GPTBot"              1;
    "~*ChatGPT-User"        1;
    "~*OAI-SearchBot"       1;
    "~*ClaudeBot"           1;
    "~*anthropic-ai"        1;
    "~*Google-Extended"     1;
    "~*Bytespider"          1;
    "~*CCBot"               1;
    "~*PerplexityBot"       1;
    "~*Applebot-Extended"   1;
    "~*DuckAssistBot"       1;
    "~*cohere-ai"           1;
    "~*Meta-ExternalAgent"  1;
    "~*Diffbot"             1;
    "~*YouBot"              1;
    "~*Amazonbot"           1;
}

server {
    listen 80;
    server_name example.com;
    root /var/www/_site;

    # Always serve robots.txt — bots can read your directives
    location = /robots.txt {
        try_files $uri =404;
        add_header X-Robots-Tag "noai, noimageai";
    }

    location / {
        if ($blocked_bot) {
            return 403;
        }
        try_files $uri $uri/ $uri.html =404;
        add_header X-Robots-Tag "noai, noimageai";
    }
}

5. Hosting comparison

PlatformHard blockX-Robots-TagNotes
NetlifyEdge Functionnetlify.toml [[headers]]Free tier; 125k edge reqs/mo
Vercelmiddleware.js (Edge)vercel.json headersNeeds next or @vercel/edge pkg
Cloudflare Pagesfunctions/_middleware.ts_headers fileGlobal CDN; free unlimited reqs
GitHub Pages❌ No server-side layer❌ No custom headersMeta tag only
Netlify _headers❌ No blocking✓ Simple _headers fileNo code required
nginx (VPS)map $http_user_agentadd_headerFull control; self-managed
S3 + CloudFrontCloudFront FunctionResponse headers policyLambda@Edge for complex logic

6. Complete eleventy.config.js reference

A minimal but complete config for an Eleventy 3.x project with robots.txt passthrough and a dedicated public/ folder for static assets.

// eleventy.config.js — Eleventy 3.x (ESM)
export default function (eleventyConfig) {

  // ── Static assets passthrough ──────────────────────────────────────────────
  // robots.txt — critical: without this it is silently dropped from _site/
  eleventyConfig.addPassthroughCopy("src/robots.txt");

  // favicon, images, fonts, etc.
  eleventyConfig.addPassthroughCopy("src/public");

  // _headers file for Netlify/Cloudflare Pages (copy to _site/_headers)
  eleventyConfig.addPassthroughCopy("src/_headers");

  // ── Watch targets ─────────────────────────────────────────────────────────
  eleventyConfig.addWatchTarget("./src/styles/");

  // ── Template languages ────────────────────────────────────────────────────
  return {
    templateFormats: ["njk", "md", "html", "liquid"],
    markdownTemplateEngine: "njk",
    htmlTemplateEngine: "njk",
    dir: {
      input: "src",
      output: "_site",
      includes: "_includes",
      layouts: "_layouts",
      data: "_data",
    },
  };
}

FAQ

Why is my robots.txt missing from _site/?

You need addPassthroughCopy("robots.txt") (or "src/robots.txt" if your input directory is "src") in your eleventy.config.js. Eleventy only outputs files it recognises as templates or that are explicitly configured as passthrough. Without this call, robots.txt is silently dropped from every build. After adding it, verify with: ls _site/robots.txt

How do I add a noai meta tag to all Eleventy pages?

Add <meta name="robots" content="noai, noimageai" /> in your base layout's <head>. For per-page control, use a robots front matter variable (robots: "noai, noimageai") and output it in the layout with {{ robots | default("noai, noimageai") }} in Nunjucks. Eleventy renders templates at build time, so the tag is in every HTML response before JavaScript runs.

Can Eleventy hard-block AI bots with a 403?

Not by itself — Eleventy outputs static files with no server process. Use a layer in front of the static output: a Netlify Edge Function, Vercel Edge Middleware (middleware.js), Cloudflare Pages Function (_middleware.ts), or nginx map block. All of these run before the static file is served, so bots receive 403 without your content.

What is the difference between Eleventy 2.x and 3.x config syntax?

Eleventy 3.x is ESM-first: the config file is eleventy.config.js using export default function(eleventyConfig) { ... }. Eleventy 2.x uses .eleventy.js with module.exports = function(eleventyConfig) { ... }. The addPassthroughCopy() API is identical in both. If you see "require is not defined", your project uses ESM but the config still uses module.exports.

How do I add X-Robots-Tag headers on Netlify?

Add a [[headers]] block in netlify.toml with for = "/*" and X-Robots-Tag = "noai, noimageai". Alternatively, create a _headers file in your publish directory (_site/) with /* on the first line and X-Robots-Tag: noai, noimageai on the second — Netlify reads this automatically.

Does noai in the meta tag affect search engine indexing?

No. The noai and noimageai directives are separate from noindex. They tell AI training crawlers (CCBot, Common Crawl, etc.) to skip the content for training datasets, but standard search bots (Googlebot, Bingbot) do not act on them for indexing decisions. You can combine them safely: content="noai, noimageai" does not suppress search indexing.

Is your site protected from AI bots?

Run a free scan to check your robots.txt, meta tags, and overall AI readiness score.

Related Guides

How to Block AI Bots on Jekyll & GitHub Pages: Complete 2026 GuideHow to Block AI Bots on Hugo: Complete 2026 Guiderobots.txt for AI Bots: Complete 2026 Guidenoai & noimageai: Block AI Training with Meta TagsHow to Block GPTBot (OpenAI)