About
Velen's public web crawler used for AI model training and general web indexing. A lesser-known crawler that has been observed across a wide range of websites.
Purpose
Web indexing and AI training
User Agent String
Mozilla/5.0 (compatible; Velenpublicwebcrawler/1.0; +https://velen.io)
How to Control in robots.txt
🚫 Block Velenpublicwebcrawler
User-agent: Velenpublicwebcrawler Disallow: /
✅ Allow Velenpublicwebcrawler
User-agent: Velenpublicwebcrawler Allow: /
Is Velenpublicwebcrawler crawling your site?
Run a free scan to check if Velen's crawler is accessing your website.
Check if Velenpublicwebcrawler is crawling YOUR site →