llms.txt
Plain Markdown file at /llms.txt that gives AI crawlers a curated index of a site's content.
Definition
llms.txt is a proposed standard (llmstxt.org) for a Markdown file served at the root of a domain that summarizes the site and lists its key pages. AI clients fetch it to get a curated, low-noise view of what's worth indexing or citing. The spec is intentionally minimal: H1 site name, optional blockquote summary, H2 sections of Markdown links — no YAML, no front matter, no nesting deeper than H2.
When to use
Add llms.txt to any site you want cited by Perplexity, ChatGPT, or Claude. It's a hint, not a contract — but it makes your most citation-worthy URLs unambiguous to retrieval crawlers.
See also
- robots.txt — Plain text file at /robots.txt that tells crawlers which paths they may or may not fetch.
- Perplexity — AI answer engine that searches the live web and returns synthesized answers with inline citations.
- ChatGPT — OpenAI's consumer LLM product — chat, browse, and run plugins on the web via GPT-class models.
- ClaudeBot — Anthropic's crawler for both Claude training data and Claude's live web-fetch tool.