Skip to content
Guides/Echo (Go)

How to Block AI Bots on Echo (Go): Complete 2026 Guide

Echo is a high-performance Go web framework built on net/http — the same standard library that Gin uses, but with a different middleware model. Echo uses the wrapper pattern: func(next echo.HandlerFunc) echo.HandlerFunc. Return echo.NewHTTPError(403, "Forbidden") to block (Layer 4), or call next(c) and set c.Response().Header().Set() to add X-Robots-Tag (Layer 3). Also used internally by PocketBase.

Wrapper pattern — not chain model

Echo middleware is a function that wraps the next handler: func(next echo.HandlerFunc) echo.HandlerFunc. Inside, you receive c echo.Context and decide whether to call next(c) (continue) or return echo.NewHTTPError(403, "Forbidden") (block). This is the same pattern as net/http's func(http.Handler) http.Handler, but operating on echo.Context instead of raw http.ResponseWriter + *http.Request. Gin uses a different model: a chain where you call c.Next()/c.Abort() on a shared context.

Protection layers

1
robots.txt — e.File("/robots.txt", "public/robots.txt") — registered before middleware
2
noai meta tag — html/template layout — {{ .Robots }} variable with default
3
X-Robots-Tag header — next(c) → c.Response().Header().Set("X-Robots-Tag", "noai, noimageai")
4
Hard 403 block — return echo.NewHTTPError(http.StatusForbidden, "Forbidden")

Layer 1: robots.txt

Create a public/ directory at your project root (alongside main.go and go.mod) and place robots.txt there. Register the file route before the bot blocker middleware.

# public/robots.txt

User-agent: *
Allow: /

User-agent: GPTBot
User-agent: ClaudeBot
User-agent: anthropic-ai
User-agent: Google-Extended
User-agent: CCBot
User-agent: Bytespider
User-agent: Applebot-Extended
User-agent: PerplexityBot
User-agent: Diffbot
User-agent: cohere-ai
User-agent: FacebookBot
User-agent: omgili
User-agent: omgilibot
User-agent: Amazonbot
User-agent: DeepSeekBot
User-agent: MistralBot
User-agent: xAI-Bot
User-agent: AI2Bot
Disallow: /

Register before bot middleware:

e := echo.New()

// Serve robots.txt BEFORE bot middleware
e.File("/robots.txt", "public/robots.txt")

// Then register bot blocker
e.Use(AiBotBlocker)

Alternatively, generate it dynamically:

e.GET("/robots.txt", func(c echo.Context) error {
    robotsTxt := `User-agent: *
Allow: /

User-agent: GPTBot
Disallow: /`
    return c.String(http.StatusOK, robotsTxt)
})

Layer 2: noai meta tag

If your Echo app renders HTML via html/template, add the noai meta tag to your base layout:

Go html/template layout (views/layout.html)

<!-- views/layout.html -->
{{- if .Robots }}
<meta name="robots" content="{{ .Robots }}">
{{- else }}
<meta name="robots" content="noai, noimageai">
{{- end }}

Render with override in Echo route handler

e.GET("/public-page", func(c echo.Context) error {
    return c.Render(http.StatusOK, "layout.html", map[string]any{
        "Robots": "index, follow",
    })
})

// Routes that omit "Robots" get the default noai value

If your Echo app is a JSON API with a separate SPA frontend, add the noai meta tag in the frontend's base layout instead.

Layers 3 & 4: echo.MiddlewareFunc

Echo middleware has the type echo.MiddlewareFunc: func(next echo.HandlerFunc) echo.HandlerFunc. Inside, you receive the Echo context and decide to block or continue.

middleware/aibot.go

package middleware

import (
	"net/http"
	"strings"

	"github.com/labstack/echo/v4"
)

var aiBotPatterns = []string{
	"gptbot", "chatgpt-user", "oai-searchbot",
	"claudebot", "anthropic-ai", "claude-web",
	"google-extended", "ccbot", "bytespider",
	"applebot-extended", "perplexitybot", "diffbot",
	"cohere-ai", "facebookbot", "meta-externalagent",
	"omgili", "omgilibot", "amazonbot",
	"deepseekbot", "mistralbot", "xai-bot", "ai2-bot",
}

var exemptPaths = map[string]bool{
	"/robots.txt":  true,
	"/sitemap.xml": true,
	"/favicon.ico": true,
}

// AiBotBlocker returns an echo.MiddlewareFunc that blocks known AI training crawlers.
func AiBotBlocker(next echo.HandlerFunc) echo.HandlerFunc {
	return func(c echo.Context) error {
		// Always pass through exempt paths
		if exemptPaths[c.Request().URL.Path] {
			return next(c)
		}

		ua := strings.ToLower(c.Request().Header.Get("User-Agent"))

		for _, pattern := range aiBotPatterns {
			if strings.Contains(ua, pattern) {
				// Layer 4: hard block — return an error, let Echo's error handler respond
				return echo.NewHTTPError(http.StatusForbidden, "Forbidden")
			}
		}

		// Layer 3: pass through, then add X-Robots-Tag to the response
		err := next(c)
		c.Response().Header().Set("X-Robots-Tag", "noai, noimageai")
		return err
	}
}

Key points

  • Blocking: return echo.NewHTTPError(http.StatusForbidden, "Forbidden") returns an error to Echo's global error handler, which converts it to a 403 response. Do NOT call next(c) — the inner handler never runs.
  • Pass-through: call err := next(c) to run the handler, then set the header with c.Response().Header().Set(). Headers set after next(c) are still written to the response as long as the body has not been flushed — safe for standard JSON/HTML responses.
  • Reading request headers: c.Request().Header.Get("User-Agent") — via *http.Request.
  • Writing response headers: c.Response().Header().Set(key, value) — via *echo.Response, which wraps http.ResponseWriter.
  • Error propagation: always return err after next(c) — if the handler returned an error, Echo's error handler needs to process it. Returning nil instead would swallow downstream errors.
  • v4 vs v5: the middleware signature is identical in Echo v4 and v5. The import path changes: github.com/labstack/echo/v4 for v4, github.com/labstack/echo/v5 for v5. PocketBase v0.22+ uses Echo v5.

Registering the middleware

package main

import (
	"log"
	"net/http"

	"github.com/labstack/echo/v4"
	"yourapp/middleware"
)

func main() {
	e := echo.New()

	// robots.txt BEFORE bot middleware
	e.File("/robots.txt", "public/robots.txt")

	// Global bot blocker — runs on every route
	e.Use(middleware.AiBotBlocker)

	// Your routes
	e.GET("/", func(c echo.Context) error {
		return c.String(http.StatusOK, "Hello, World!")
	})
	e.GET("/api/data", func(c echo.Context) error {
		return c.JSON(http.StatusOK, map[string]string{"status": "ok"})
	})

	log.Fatal(e.Start(":1323"))
}

Echo runs middleware in FIFO order — the first e.Use() call is the outermost wrapper and runs first. Register the bot blocker before auth, CORS, and body parsing middlewares.

Route-group blocking

To protect only API routes while leaving the public frontend indexable:

// Public routes — no bot blocking
e.GET("/", homeHandler)
e.GET("/about", aboutHandler)

// API routes — bot blocker applied to this group
api := e.Group("/api")
api.Use(middleware.AiBotBlocker)
api.GET("/products", productsHandler)
api.GET("/users", usersHandler)
api.POST("/orders", ordersHandler)

e.Group("/api") creates a sub-router; group.Use() adds middleware only for that group. Multiple middlewares can be chained: api.Use(echomiddleware.CORS(), middleware.AiBotBlocker).

Echo v4 vs v5 — import path only

The middleware code above works for both Echo v4 and v5. The only difference is the import path:

Echo v4 (current stable)

import "github.com/labstack/echo/v4"

Echo v5 (used by PocketBase v0.22+)

import "github.com/labstack/echo/v5"

If you use PocketBase and add middleware via app.OnBeforeServe().Add(), Echo's router is already set up — use e.Router.Use(aiBotMiddleware) with the v5 import path. See the PocketBase guide for details.

Echo vs Gin vs net/http

All three run on net/http, but the middleware patterns differ:

Echo (wrapper pattern, echo.Context)

func AiBotBlocker(next echo.HandlerFunc) echo.HandlerFunc {
    return func(c echo.Context) error {
        if isBot(c.Request().Header.Get("User-Agent")) {
            return echo.NewHTTPError(403, "Forbidden")
        }
        err := next(c)
        c.Response().Header().Set("X-Robots-Tag", "noai, noimageai")
        return err
    }
}

Gin (chain model, gin.Context)

func AiBotBlocker() gin.HandlerFunc {
    return func(c *gin.Context) {
        if isBot(c.GetHeader("User-Agent")) {
            c.AbortWithStatus(403)
            return
        }
        c.Next()
        c.Header("X-Robots-Tag", "noai, noimageai")
    }
}

net/http (wrapper pattern, http.Handler)

func AiBotBlocker(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        if isBot(r.Header.Get("User-Agent")) {
            http.Error(w, "Forbidden", 403)
            return
        }
        w.Header().Set("X-Robots-Tag", "noai, noimageai")
        next.ServeHTTP(w, r)
    })
}

Echo and net/http both use the wrapper pattern — a function that takes the next handler and returns a new handler. Echo replaces http.Handler with echo.HandlerFunc and http.ResponseWriter + *http.Request with echo.Context. Gin uses a chain model instead: c.Next()/c.Abort() on a shared context passed to every handler in order.

Verification

# Layer 1 — robots.txt (served before middleware)
curl http://localhost:1323/robots.txt

# Layer 3 — X-Robots-Tag on legitimate request
curl -I http://localhost:1323/api/products
# Expected: X-Robots-Tag: noai, noimageai

# Layer 4 — hard block on bot user-agent
curl -A "Mozilla/5.0 (compatible; GPTBot/1.0; +https://openai.com/gptbot)" \
  -I http://localhost:1323/api/products
# Expected: HTTP/1.1 403 Forbidden

# robots.txt must pass through even for bot UAs
curl -A "GPTBot" -I http://localhost:1323/robots.txt
# Expected: HTTP/1.1 200 OK

FAQ

How does Echo middleware differ from Gin middleware?

Echo uses the wrapper pattern: func(next echo.HandlerFunc) echo.HandlerFunc. You call next(c) to continue or return an error to block. Gin uses a chain model: middleware is func(*gin.Context) and you call c.Next() to continue or c.AbortWithStatus() to block. Echo's wrapper is closer to net/http's func(http.Handler) http.Handler but uses echo.Context.

How do I return a 403 in Echo middleware?

Return echo.NewHTTPError(http.StatusForbidden, "Forbidden"). Echo's global error handler converts this to a 403 HTTP response. Do not call next(c) after this — just return the error. The inner handler and all downstream middleware do not run.

Where does robots.txt go in an Echo project?

Register before the bot blocker: e.File("/robots.txt", "public/robots.txt"). Echo processes routes in registration order, so the file handler runs before the bot blocker middleware. For dynamic generation, e.GET("/robots.txt", func(c echo.Context) error { return c.String(200, robotsTxt) }) also works.

Can I set headers after next(c) in Echo?

Yes — call err := next(c), then set the header with c.Response().Header().Set(). This works because Echo buffers response headers until the first body write. Safe for standard JSON/HTML responses. For streaming responses (SSE, chunked), set headers before next(c) to guarantee delivery.

Does Echo v5 middleware work differently from v4?

The middleware signature and echo.Context API are the same. The only difference is the import path: github.com/labstack/echo/v4 vs /v5. PocketBase v0.22+ uses Echo v5. If you're extending PocketBase, use the v5 import and the e.Router.Use() pattern from the PocketBase guide.

Is your site protected from AI bots?

Run a free scan to check your robots.txt, meta tags, and overall AI readiness score.