Robots.txt Generator

Choose a preset or configure each AI crawler individually. Decide who gets to train on your content and who gets to cite it.

🚀

Maximize AI Visibility

Allow all AI bots — training and search. Best for GEO reach.

🎯

AI Search, No Training

Let AI cite you in answers, block bots that train models.

🚫

Block All AI Bots

Traditional search only — Google, Bing indexing only.

🌐

Allow Everything

No restrictions on any crawler at all.

AI Crawlers Allow / Block / Skip

Traditional Crawlers

Custom Rules

Disallow Paths (applies to all unlisted crawlers)

One path per line

Sitemap URL (optional)

📄 robots.txt