Skip to content
Adrythm
Robots.txt generator
Ad management
AI solutions
Websites
SEO and AI search
Custom software
Resources
About
Book 30 minutes
Free tools
Robots.txt
generator
Enable JavaScript to generate robots.txt.
Crawler rules
*
Group 1
User-agent
Disallow paths · one per line
Allow exceptions · one per line
Add crawler group
Reset
Sitemap URLs · one per line
robots.txt
User-agent: * Disallow:
Copy robots.txt
Download robots.txt
Test a path · ASCII only
Crawler product token
ASCII path and query
Allowed by these rules
* · No matching rule
Google robots.txt specification
