Documentation
Guides on managing AI bots, optimizing for AI search engines, and monitoring your AI presence.
Understanding AI Bots
What are GPTBot, ClaudeBot, and PerplexityBot? Why do they crawl your site and what do they do with your content?
View Bot Directory →AI Readiness Score
How Shadow calculates your score across SEO, security, accessibility, performance, and structured data — and what each category means.
Check Your Score →robots.txt Generator
Interactive tool to configure your robots.txt for any AI crawler. One-click presets, per-bot rules, copy-paste output. Free.
Open Generator →robots.txt Analyzer
Paste your existing robots.txt and see which AI bots you're allowing or blocking. Flags critical issues like blocking Googlebot.
Analyze My robots.txt →What AI Thinks About You
Shadow queries ChatGPT, Claude, Perplexity, and Gemini about your brand. Here's how it works and what you can do about wrong information.
Scan Your Brand →In-Depth Guides
Long-form guides on robots.txt, AI meta tags, blocking strategies, and more — with copy-paste code examples.
Browse all guides →Start with a free scan
See your AI readiness score, which bots are accessing your site, and what to fix first.
Check Your Site Free →