How to Block AI Bots on NestJS: Complete 2026 Guide

Methods at a glance

NestJS request pipeline

Understanding where each layer fires determines the right tool for bot blocking.

1. robots.txt — @nestjs/serve-static

Register ServeStaticModule in AppModule. It serves all files in your public/ directory at the root path. public/robots.txt becomes accessible at /robots.txt with no controller needed.

public/robots.txt

User-agent: GPTBot
Disallow: /

User-agent: ChatGPT-User
Disallow: /

User-agent: OAI-SearchBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Claude-Web
Disallow: /

User-agent: anthropic-ai
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: Bytespider
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: PerplexityBot
Disallow: /

User-agent: Applebot-Extended
Disallow: /

User-agent: *
Allow: /

app.module.ts — ServeStaticModule

import { Module } from '@nestjs/common';
import { ServeStaticModule } from '@nestjs/serve-static';
import { join } from 'node:path';

@Module({
  imports: [
    ServeStaticModule.forRoot({
      rootPath: join(__dirname, '..', 'public'),
      // Serve at root path — /robots.txt, /favicon.ico, etc.
      serveRoot: '/',
      // Don't interfere with API routes
      exclude: ['/api/(.*)'],
    }),
    // ... other modules
  ],
})
export class AppModule {}

// Install: npm install @nestjs/serve-static

Alternative: dedicated controller

import { Controller, Get, Res } from '@nestjs/common';
import { Response } from 'express';
import { ConfigService } from '@nestjs/config';

const ROBOTS_PROD = `User-agent: GPTBot
Disallow: /
User-agent: ChatGPT-User
Disallow: /
User-agent: OAI-SearchBot
Disallow: /
User-agent: ClaudeBot
Disallow: /
User-agent: anthropic-ai
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: Bytespider
Disallow: /
User-agent: CCBot
Disallow: /
User-agent: PerplexityBot
Disallow: /
User-agent: *
Allow: /`;

const ROBOTS_DEV = 'User-agent: *\nDisallow: /';

@Controller('robots.txt')
export class RobotsController {
  constructor(private readonly config: ConfigService) {}

  @Get()
  robots(@Res() res: Response): void {
    const isProd = this.config.get<string>('NODE_ENV') === 'production';
    res.type('text/plain').send(isProd ? ROBOTS_PROD : ROBOTS_DEV);
  }
}

2. app.use() in main.ts — simplest global block

NestJS runs on Express by default. app.use() in main.ts registers an Express middleware that fires before any NestJS routing or DI resolution — the earliest possible interception point.

import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import type { Request, Response, NextFunction } from 'express';

const BLOCKED_UAS =
  /GPTBot|ChatGPT-User|OAI-SearchBot|ClaudeBot|Claude-Web|anthropic-ai|Google-Extended|Bytespider|CCBot|PerplexityBot|Applebot-Extended/i;

async function bootstrap() {
  const app = await NestFactory.create(AppModule);

  // Register BEFORE app.listen() — fires before all NestJS logic
  app.use((req: Request, res: Response, next: NextFunction) => {
    // Always allow robots.txt
    if (req.path === '/robots.txt') return next();

    const ua = req.headers['user-agent'] ?? '';
    if (BLOCKED_UAS.test(ua)) {
      return res.status(403).type('text/plain').send('Forbidden');
    }
    next();
  });

  await app.listen(3000);
}

bootstrap();

3. NestMiddleware — DI-aware middleware

NestMiddleware is the NestJS way to write middleware — it participates in the dependency injection system, so you can inject ConfigService or any provider. Register it in AppModule.configure() using the MiddlewareConsumer.

middleware/block-ai-bots.middleware.ts

import { Injectable, NestMiddleware } from '@nestjs/common';
import type { Request, Response, NextFunction } from 'express';

const BLOCKED_UAS =
  /GPTBot|ChatGPT-User|OAI-SearchBot|ClaudeBot|Claude-Web|anthropic-ai|Google-Extended|Bytespider|CCBot|PerplexityBot|Applebot-Extended/i;

@Injectable()
export class BlockAiBotsMiddleware implements NestMiddleware {
  use(req: Request, res: Response, next: NextFunction): void {
    // Exempt robots.txt before UA check
    if (req.path === '/robots.txt') {
      return next();
    }

    const ua = req.headers['user-agent'] ?? '';
    if (BLOCKED_UAS.test(ua)) {
      res.status(403).type('text/plain').send('Forbidden');
      return;
    }

    next();
  }
}

app.module.ts — configure() registration

import { MiddlewareConsumer, Module, NestModule, RequestMethod } from '@nestjs/common';
import { BlockAiBotsMiddleware } from './middleware/block-ai-bots.middleware';

@Module({
  // ... imports, controllers, providers
})
export class AppModule implements NestModule {
  configure(consumer: MiddlewareConsumer): void {
    consumer
      .apply(BlockAiBotsMiddleware)
      // Apply to all routes except /robots.txt
      .exclude({ path: 'robots.txt', method: RequestMethod.GET })
      // Apply to all other paths
      .forRoutes('*');
  }
}

.exclude() is an alternative to the path check inside the middleware. Both work — the inline check is slightly more explicit about intent.

4. Guard (CanActivate) — per-controller blocking

Guards fire after route matching — later than Middleware but with full access to the ExecutionContext (route metadata, decorators). Use Guards when you need to block AI bots only on specific controllers and want to use the decorator pattern.

guards/block-ai-bots.guard.ts

import { CanActivate, ExecutionContext, Injectable } from '@nestjs/common';
import { HttpException, HttpStatus } from '@nestjs/common';
import type { Request } from 'express';

const BLOCKED_UAS =
  /GPTBot|ChatGPT-User|OAI-SearchBot|ClaudeBot|Claude-Web|anthropic-ai|Google-Extended|Bytespider|CCBot|PerplexityBot|Applebot-Extended/i;

@Injectable()
export class BlockAiBotsGuard implements CanActivate {
  canActivate(context: ExecutionContext): boolean {
    const request = context.switchToHttp().getRequest<Request>();
    const ua = request.headers['user-agent'] ?? '';

    if (BLOCKED_UAS.test(ua)) {
      throw new HttpException('Forbidden', HttpStatus.FORBIDDEN);
    }

    return true;
  }
}

Apply globally in main.ts

import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import { BlockAiBotsGuard } from './guards/block-ai-bots.guard';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);

  // Global guard — applies to all controllers
  // Note: fires after route matching, unlike Middleware
  app.useGlobalGuards(new BlockAiBotsGuard());

  await app.listen(3000);
}

bootstrap();

Apply per-controller with decorator

import { Controller, Get, UseGuards } from '@nestjs/common';
import { BlockAiBotsGuard } from '../guards/block-ai-bots.guard';

@Controller('api')
@UseGuards(BlockAiBotsGuard) // Only this controller is protected
export class ApiController {
  @Get('content')
  getContent() {
    return { data: 'protected' };
  }
}

// No guard on this controller — public
@Controller()
export class AppController {
  @Get()
  home() {
    return 'Hello';
  }
}

5. Interceptor — X-Robots-Tag response header

Interceptors wrap the handler execution and can modify responses. Use an Interceptor to add X-Robots-Tag to all responses — it fires after the handler, so it sees the full response before it's sent.

interceptors/x-robots-tag.interceptor.ts

import {
  CallHandler,
  ExecutionContext,
  Injectable,
  NestInterceptor,
} from '@nestjs/common';
import { Observable, tap } from 'rxjs';
import type { Response } from 'express';

@Injectable()
export class XRobotsTagInterceptor implements NestInterceptor {
  intercept(context: ExecutionContext, next: CallHandler): Observable<unknown> {
    return next.handle().pipe(
      tap(() => {
        const response = context.switchToHttp().getResponse<Response>();
        const request = context.switchToHttp().getRequest();
        // Don't set X-Robots-Tag on robots.txt itself
        if (request.path !== '/robots.txt') {
          response.setHeader('X-Robots-Tag', 'noai, noimageai');
        }
      })
    );
  }
}

Register globally in main.ts

app.useGlobalInterceptors(new XRobotsTagInterceptor());

6. noai meta tag with @nestjs/view

NestJS supports server-side rendering via @nestjs/view with Handlebars, Pug, or EJS. Pass a blockAiTraining flag from your controller to the template.

views/layout.hbs — Handlebars

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  {{#if blockAiTraining}}
  <meta name="robots" content="noai, noimageai">
  {{/if}}
  <title>{{title}}</title>
</head>
<body>
  {{{body}}}
</body>
</html>

Controller with @Render()

import { Controller, Get, Render } from '@nestjs/common';

@Controller()
export class AppController {
  @Get()
  @Render('layout')
  home() {
    return {
      title: 'Home',
      body: '<h1>Welcome</h1>',
      blockAiTraining: false, // Allow AI training on public pages
    };
  }

  @Get('articles')
  @Render('layout')
  articles() {
    return {
      title: 'Articles',
      body: '<h1>Our Articles</h1>',
      blockAiTraining: true, // Block AI training on content pages
    };
  }
}

7. nginx — reverse proxy hard block

map $http_user_agent $block_ai_bot {
    default              0;
    "~*GPTBot"           1;
    "~*ChatGPT-User"     1;
    "~*OAI-SearchBot"    1;
    "~*ClaudeBot"        1;
    "~*Claude-Web"       1;
    "~*anthropic-ai"     1;
    "~*Google-Extended"  1;
    "~*Bytespider"       1;
    "~*CCBot"            1;
    "~*PerplexityBot"    1;
    "~*Applebot-Extended" 1;
}

server {
    listen 80;
    server_name yourdomain.com;

    location = /robots.txt {
        alias /var/www/public/robots.txt;
        add_header Content-Type "text/plain";
    }

    location / {
        if ($block_ai_bot) {
            return 403;
        }
        proxy_pass http://127.0.0.1:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

8. Docker multi-stage build

FROM node:22-alpine AS base
WORKDIR /app
COPY package.json package-lock.json ./

FROM base AS deps
RUN npm ci

FROM base AS builder
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN npm run build

FROM node:22-alpine AS runner
WORKDIR /app
ENV NODE_ENV=production

COPY --from=builder /app/dist ./dist
COPY --from=deps /app/node_modules ./node_modules
COPY public ./public

EXPOSE 3000
CMD ["node", "dist/main.js"]

NestJS layers compared

FAQ