How is this different from Google Analytics?

Google Analytics shows you traffic. Shadow shows you traffic, AI bot activity, what AI platforms say about your brand, AND tells you what to do about all of it. It's analytics + AI intelligence + action steps in one tool.

Do I need to install anything?

For basic monitoring (bot detection, AI perception, readiness score) — nope, just enter your URL. For full visitor analytics (clicks, behavior, sessions), add one script tag. One-click integrations for Vercel, Shopify, WordPress, and more.

Will it slow down my site?

No. The script is under 5KB and loads async. Zero impact on page speed or Core Web Vitals. External monitoring has literally no impact — it watches from the outside.

What AI bots does Shadow detect?

All of them. GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Bytespider, Amazonbot, and dozens more. The Shadow Network means new bots get identified across all users instantly.

What do you mean by "actionable steps"?

Shadow doesn't just show you graphs. It says things like: "ChatGPT has your pricing wrong — add structured data to /pricing to fix it" or "Your bounce rate on /features is 68% — here's why and what to change." Specific, do-it-today recommendations.

Can Shadow block bots?

Shadow is a telescope, not a shield. It shows you who's visiting and what AI says about you. It generates block rules and robots.txt configs you can apply — but it doesn't intercept traffic.

Yes. Shadow never collects PII. IP addresses are hashed after classification. No cookies on your visitors. All Shadow Network data is anonymized. GDPR compliant by design.

How do I block Ai2Bot?

Add this to your robots.txt file: User-agent: Ai2Bot / Disallow: /

Does Ai2Bot respect robots.txt?

Yes, Ai2Bot by Allen Institute for AI (Ai2) respects robots.txt directives.

AI Training Respects robots.txt

Ai2Bot

by Allen Institute for AI (Ai2) First seen: 2022-10

About

The Allen Institute for AI's research crawler, used to build Dolma — one of the largest open datasets for training language models. Unlike commercial crawlers, Ai2Bot's data is publicly released for research. Behind some of the most widely-used open-source LLM training corpora.

Purpose

Open research dataset collection for AI model training

User Agent String

Mozilla/5.0 (compatible; Ai2Bot/1.0; +https://allenai.org/crawler)

How to Control in robots.txt

🚫 Block Ai2Bot

User-agent: Ai2Bot
Disallow: /

✅ Allow Ai2Bot

User-agent: Ai2Bot
Allow: /

Complete Guide: How to Block Ai2Bot

Server-level blocking, nginx configs, Cloudflare rules, Next.js middleware, and more →

Is Ai2Bot crawling your site?

Run a free scan to check if Allen Institute for AI (Ai2)'s crawler is accessing your website.

Check if Ai2Bot is crawling YOUR site →