About
Baidu's web crawler powering China's most used search engine and its AI platform ERNIE Bot. With over 70% search market share in China, Baiduspider is the gateway to 1.4 billion potential users. Its crawl data also feeds Baidu's Wenxin Yiyan (ERNIE Bot) large language model for AI-generated answers.
Purpose
Baidu search indexing and ERNIE Bot AI training
User Agent String
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
How to Control in robots.txt
🚫 Block Baiduspider
User-agent: Baiduspider Disallow: /
✅ Allow Baiduspider
User-agent: Baiduspider Allow: /
Is Baiduspider crawling your site?
Enter your URL below — scan takes under 5 seconds.
Free · No signup · Instant results