Robots.txt Tester
Test whether a URL path is allowed for a selected crawler using robots.txt rules.
About Robots.txt Tester
Paste robots.txt rules, choose a crawler and test a path. The evaluator follows the most specific matching user-agent group and the longest matching allow or disallow rule.
Helpful context
Does this fetch my live robots.txt file?
No. Paste the file contents so the test stays client-side and avoids cross-origin restrictions.
Does robots.txt prevent indexing?
Not reliably. Robots.txt controls crawling. Use a noindex directive on an accessible page when you need to prevent indexing.