SEO

robots.txt generator

Build your rules, copy the file, and test whether a URL is crawlable — for any bot.

Start from:

Sitemaps

robots.txt

Test a URL

User-agent matching is simplified (case-insensitive exact token, else *) — fine for files you author yourself.