Run a free check for robots.txt, sitemap.xml, llms.txt, AI crawler rules, link headers, and markdown access.
No login and no database. Results stay in this tab session until the tab closes.
Agent readiness report
session onlyWhat the audit checks
We parse User-agent rules, Sitemap lines, and policy signals.
We check /llms.txt and optional /llms-full.txt for useful markdown.
We look for sitemap.xml and RFC 8288 Link headers.
We request markdown from the homepage and report what changed.
No vague AI score. Each row shows what we fetched, what we found, and why it passed or failed.
Presence, status, valid directives, and sitemap hints.
Robots hints plus common sitemap paths and XML structure.
Markdown file with links AI agents can follow.
GPTBot, Claude-Web, Google-Extended, PerplexityBot, and more.
Content-Signal directives for search, input, and training policy.
Homepage Link headers used by crawlers and agents.
Whether the homepage returns markdown when agents ask for it.
Every check expands to show request, response, and conclusion.
Why it matters
Useful SEO now includes the files and headers AI crawlers check before they read, cite, or navigate your website.
A 200 response is not enough. The audit reads the file body and tells you whether AI crawlers can use it.
Most generic crawlers skip llms.txt. This scan treats it as a first-class discovery file.
Open any result card to see what was fetched and why the check passed or failed.
Paste a URL and get a simple report for robots.txt, sitemap.xml, llms.txt, AI bot rules, and content access.