AI Guide
AI crawlability checklist
If your content is supposed to be publicly reachable, this checklist catches the most common technical reasons it still ends up invisible.
Core checks
- Homepage returns a public 200 response over HTTPS.
- Important content is not hidden behind auth, challenge pages or heavy client-side dependence.
- robots.txt exists and is readable.
- Tracked AI bot rules are intentionally allowed or intentionally blocked.
- Sitemap is published and linked from robots.txt.
- llms.txt, if used, is publicly reachable and current.
Operational checks
The technical file can look clean while the real publishing workflow stays messy. The operational side matters too: internal linking, stable URLs, fast responses and documented policy decisions all make later debugging easier.
What this checklist is for
This is for public crawlability. It does not try to answer whether every model will cite you, summarize you accurately, or send traffic back. It is the “can they fetch the basics?” checklist.
When to use the PDF
The PDF becomes more useful when someone else needs to act on the result. It turns a vague “AI visibility issue” into a short list of checks and next fixes that an engineer, SEO or content owner can work from.