Free technical SEO tool

Robots.txt Checker

Check whether a website exposes a robots.txt file, what it tells crawlers, and whether Googlebot appears allowed to access the public page.

Free · no account · only bounded public responses are read

What this check does

It fetches the public robots.txt file, looks for sitemap declarations, and applies the most specific observed Allow or Disallow rule for Googlebot to the path you entered.

What this check does not prove

Robots permission is not an index-status check. An allowed path may still be excluded, canonicalized elsewhere, or not yet processed by Google.

Robots.txt is only one part of technical SEO.

Also inspect sitemap discovery, page directives, canonicals and the initial public response.

Run full SEO audit · Check an XML sitemap