About
The crawler for You.com, an AI-powered search engine. Indexes web content to deliver AI-summarized search results with source attribution.
Purpose
AI search engine indexing
User Agent String
Mozilla/5.0 (compatible; YouBot/1.0; +https://about.you.com/youbot)
How to Control in robots.txt
🚫 Block YouBot
User-agent: YouBot Disallow: /
✅ Allow YouBot
User-agent: YouBot Allow: /
Complete Guide: How to Block YouBot
Server-level blocking, nginx configs, Cloudflare rules, Next.js middleware, and more →
Is YouBot crawling your site?
Run a free scan to check if You.com's crawler is accessing your website.
Check if YouBot is crawling YOUR site →