Robots.txt Generator

Generate a valid robots.txt in seconds — with the critical "block everything" warning built in.

User agent

Crawl-delay (optional)

Allow paths

Disallow paths

Sitemap URL

Runs entirely in your browser — nothing is stored or sent. Part of Webot's free tools.