Public access
PASSHomepage responds publicly.
AI Audit
Test whether your public site is actually reachable by AI crawlers.
Measure whether AI systems can access, interpret and discover your public content, with evidence from live crawler probes, page structure, policy files and discovery signals.
GPTBot · ChatGPT-User · Google-Extended · ClaudeBot · PerplexityBot · CCBot
Free · No email required · Results in seconds
AuditLabHQ only reads public website signals. Nothing is modified.
Example audit
AI crawlability summary
78/100 B
Example severity: medium
robots.txt
PASSCrawler policy is reachable.
AI bot policy
WARNSome crawler rules need review.
Extractable content
PASSUseful HTML content is exposed.
Structured data
WARNOrganization signals are incomplete.
llms.txt
NOT VERIFIEDNo useful file was evidenced.
9 categories
Access, policy, live probes, content, schema and discovery.
Fast scan
Useful before content, SEO or distribution assumptions turn into guesswork.
PDF option
A one-page decision brief or a detailed evidence and implementation report.
Recommended
What makes the audit useful
From “the bot is allowed” to evidence your team can act on.
Intent in robots.txt
We resolve wildcard and explicit rules for 12 search, training and user-triggered AI agents.
Observed crawler response
Live probes reveal when CDN, WAF or challenge behavior disagrees with the published policy.
Content AI can interpret
We inspect initial HTML, headings, canonical/indexation, internal trust links, JSON-LD and discovery files.
What the audit checks
- Whether the homepage returns a public HTTP response without obvious access friction.
- Whether robots.txt is present and explicit enough to audit AI bot policy.
- Whether 12 tracked AI agents look allowed or blocked at the root path.
- Whether real AI user-agent probes receive usable HTML through your CDN or WAF.
- Whether the HTML exposes strong content, canonical/indexation signals and JSON-LD.
- Whether your sitemap is valid and llms.txt contains useful guidance.
Guides for the same problems the scan checks
robots.txt for AI bots
See how robots.txt shapes AI crawler access and where teams usually over-block themselves.
How to allow AI bots
A practical guide to allowing public AI crawlers without turning the site into a free-for-all.
llms.txt explained
What llms.txt can and cannot do, and why it is not a replacement for robots.txt.
AI crawlability checklist
A short pre-launch checklist for teams that want discoverable public content.
Why AI crawlers cannot access your site
A breakdown of the most common causes: homepage errors, bot blocking, missing files and ambiguity.
FAQ
Does this guarantee inclusion in AI answers?
No. It checks crawlability signals, not ranking, retrieval quality, citation policy or model behavior.
Does llms.txt replace robots.txt?
No. robots.txt is the more established crawler control layer. llms.txt is best treated as supplementary guidance.
Should every AI bot be allowed?
Not necessarily. The useful decision is intentional access, not blanket access or blanket blocking by default.