How is this different from Google Analytics?

Google Analytics shows you traffic. Shadow shows you traffic, AI bot activity, what AI platforms say about your brand, AND tells you what to do about all of it. It's analytics + AI intelligence + action steps in one tool.

Do I need to install anything?

For basic monitoring (bot detection, AI perception, readiness score) — nope, just enter your URL. For full visitor analytics (clicks, behavior, sessions), add one script tag. One-click integrations for Vercel, Shopify, WordPress, and more.

Will it slow down my site?

No. The script is under 5KB and loads async. Zero impact on page speed or Core Web Vitals. External monitoring has literally no impact — it watches from the outside.

What AI bots does Shadow detect?

All of them. GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Bytespider, Amazonbot, and dozens more. The Shadow Network means new bots get identified across all users instantly.

What do you mean by "actionable steps"?

Shadow doesn't just show you graphs. It says things like: "ChatGPT has your pricing wrong — add structured data to /pricing to fix it" or "Your bounce rate on /features is 68% — here's why and what to change." Specific, do-it-today recommendations.

Can Shadow block bots?

Shadow is a telescope, not a shield. It shows you who's visiting and what AI says about you. It generates block rules and robots.txt configs you can apply — but it doesn't intercept traffic.

Yes. Shadow never collects PII. IP addresses are hashed after classification. No cookies on your visitors. All Shadow Network data is anonymized. GDPR compliant by design.

AI Bot Directory

Every AI crawler, identified. Know what's visiting your site, who operates it, and whether it respects your rules.

61 bots tracked·Updated Mar 2026·robots.txt Generator →

GPTBot

OpenAI

AI model training data collection

AI TrainingSince 2023-08

ChatGPT-User

OpenAI

Real-time web browsing for ChatGPT users

AI AssistantSince 2023-04

OAI-SearchBot

OpenAI

AI-powered search indexing

AI SearchSince 2024-07

ClaudeBot

Anthropic

AI model training data collection

AI TrainingSince 2023-12

PerplexityBot

Perplexity AI

AI search engine indexing and real-time retrieval

AI SearchSince 2023-06

Google-Extended

Google

AI model training (Gemini, Bard)

AI TrainingSince 2023-09

Gemini

Google

Real-time web browsing for Gemini users

AI AssistantSince 2024-02

Bytespider

ByteDance

AI model training and content indexing

AI TrainingSince 2022-01

CCBot

Common Crawl

Open web dataset for AI training and research

AI TrainingSince 2011-01

Amazonbot

Amazon

AI assistant answers and product indexing

AI ShoppingSince 2022-06

AppleBot-Extended

Apple

AI model training for Apple Intelligence

AI TrainingSince 2024-06

FacebookBot

Meta-ExternalAgent

cohere-ai

Cohere

Enterprise AI model training

AI TrainingSince 2023-08

YouBot

You.com

AI search engine indexing

AI SearchSince 2022-09

Diffbot

Structured data extraction and knowledge graph building

OtherSince 2012-01

PetalBot

Huawei (Petal Search)

Search engine indexing for Petal Search

SEOSince 2020-06

Barkrowler

Babbar

SEO analysis and backlink mapping

SEOSince 2020-01

Timpibot

Timpi

Decentralized search indexing

AI SearchSince 2023-01

Seekr

Seekr Technologies

AI-scored search and content evaluation

AI SearchSince 2023-03

Kangaroo Bot

Kangaroo LTD

Content indexing and data analytics

OtherSince 2023-06

Velenpublicwebcrawler

Velen

Web indexing and AI training

AI TrainingSince 2023-09

Omgili

Omgili (Webz.io)

Forum and discussion content crawling for AI datasets

AI TrainingSince 2010-01

ICC-Crawler

NICT (Japan)

Multilingual AI research and translation training

AI TrainingSince 2015-01

Brightbot

Bright Data

Large-scale data collection for AI and analytics

OtherSince 2021-01

Scrapy

Open Source (Zyte)

General-purpose web scraping framework

OtherSince 2008-01

xAI-Bot

xAI (Elon Musk)

AI model training for Grok — real-time knowledge and news indexing

AI TrainingSince 2024-02

DuckAssistBot

DuckDuckGo

Real-time page fetching for DuckAssist AI summaries

AI SearchSince 2023-03

Bingbot

Microsoft

Search indexing for Bing + real-time grounding for Microsoft Copilot

AI SearchSince 2010-06

Ai2Bot

Allen Institute for AI (Ai2)

Open research dataset collection for AI model training

AI TrainingSince 2022-10

MistralBot

Mistral AI

AI model training data collection

AI TrainingSince 2024-01

Googlebot

Google

Search indexing and Google AI Overviews data sourcing

AI SearchSince 1997-01

Applebot

Apple

Siri answers, Spotlight search, and Safari web suggestions

AI AssistantSince 2015-03

AhrefsBot

Ahrefs

Backlink analysis and SEO intelligence platform data collection

SEOSince 2011-01

SemrushBot

Semrush

SEO competitive intelligence and site audit data collection

SEOSince 2010-01

YandexBot

Yandex

Yandex search indexing and YaGPT AI training

AI SearchSince 2005-01

Baiduspider

Baidu

Baidu search indexing and ERNIE Bot AI training

AI SearchSince 2001-01

BraveBot

Brave

Independent search indexing for Brave Search

AI SearchSince 2021-06

NaverBot

Naver

Naver search indexing and HyperCLOVA X AI training

AI SearchSince 2008-01

HuggingFaceBot

Hugging Face

AI dataset collection for Hugging Face Hub

AI TrainingSince 2021-03

DuckDuckBot

DuckDuckGo

Search indexing and DuckAssist AI answer support

AI SearchSince 2010-04

LinkedInBot

LinkedIn (Microsoft)

Link preview metadata and LinkedIn AI features

SocialSince 2011-01

archive.org_bot

Internet Archive

Web preservation and Wayback Machine archiving

OtherSince 1996-10

TurnitinBot

Turnitin

Academic plagiarism detection and AI content indexing

OtherSince 2005-06

iaskspider

iAsk.ai

AI-powered search engine indexing

AI SearchSince 2023-01

Sogou

Tencent

Sogou search indexing and Tencent Hunyuan AI training

AI SearchSince 2004-08

DataForSeoBot

DataForSEO

SEO data collection and AI-powered marketing research

SEOSince 2018-01

MJ12bot

Majestic

Link intelligence database and AI SEO tool support

SEOSince 2004-01

rogerbot

Moz

SEO metrics and AI content tool support

SEOSince 2009-06

DeepSeekBot

DeepSeek

AI model training data collection

AI TrainingSince 2024-06

Firecrawl

Mendable / Firecrawl

Web scraping for AI agent applications

AI AssistantSince 2024-01

JinaBot

Jina AI

LLM-ready content extraction and AI embedding

AI AssistantSince 2023-11

ExaBot

Exa AI

Neural search indexing for AI applications

AI SearchSince 2024-02

MojeekBot

Mojeek

Privacy-focused independent search index

OtherSince 2004-01

ApifyBot

Apify

Web scraping and AI training data collection

AI TrainingSince 2015-01

QwenBot

Alibaba Cloud

AI model training data collection

AI TrainingSince 2024-01

YandexGPT

Yandex

AI search summaries and LLM training

AI SearchSince 2023-09

img2dataset

HuggingFace / Community

Bulk image dataset collection for AI training

AI TrainingSince 2021-06

NewsPlease

Community / Various

News and media dataset collection for NLP/LLM training

AI TrainingSince 2017-01

Slurp

Yahoo / Verizon Media

Web indexing for Yahoo Search and partner properties

SEOSince 1996-01

Twitterbot

X (formerly Twitter)

Link card previews, Open Graph metadata fetching, and AI training data collection for Grok

AI TrainingSince 2012-06

Are these bots crawling your site?

Run a free scan to see which AI bots are accessing your website right now.

Check Your Site →