Robots.txt Checker
Fetch and validate robots.txt files. Check allow/disallow rules and sitemap declarations.
Framework ready — register a checker in lib/checkers/checkers/robots-txt-checker.ts to enable scanning.
Enter URLs
Paste URLs, upload a CSV/TXT file, or drag & drop. Up to 500 URLs per scan.
0 / 500 URLs
No URLs yet
Enter URLs above or upload a CSV/TXT file to get started.