Is llms.txt an official standard?

No. It is a community proposal documented at llmstxt.org. As of mid-2026 no major model provider commits to fetching it, but adoption keeps growing because the cost is one static file and the downside is zero.

How is llms.txt different from robots.txt?

robots.txt controls access: which paths crawlers may fetch. llms.txt aids comprehension: which pages are worth fetching and what each contains. One excludes, the other curates; sites benefit from both.

Do I also need llms-full.txt?

Optional. It inlines full content for agents that want everything in one fetch, which suits documentation sites. A store or SaaS marketing site gets most of the value from a well-curated llms.txt alone.

llms.txt: a map of your site for AI agents

Updated 2026-07-13 · 3 min read · comprehension pillar

llms.txt is a plain markdown file served at the root of your site, proposed by Jeremy Howard of Answer.AI in September 2024. The idea is simple: language models work with limited context, and your HTML spends most of it on navigation, scripts, and boilerplate. llms.txt hands an agent the short version: what this site is, and which pages matter, each with a one-line description.

The format is deliberately minimal. An H1 with the site name, a blockquote summarizing what you do, then H2 sections containing markdown link lists in the form [Title](url): description. An optional section named Optional marks links an agent can skip when context is tight. A companion file, llms-full.txt, can inline full page content for sites that want it.

Why a curated map earns money

An agent arriving at your site has a fetch budget of a handful of requests. Without a map it spends them guessing: homepage, a nav link, maybe your blog. With llms.txt it spends them on the pages you chose: pricing, shipping policy, your catalog, the docs page that answers the question. Answers about your prices and terms then come from your canonical pages instead of a third party's summary.

Honesty about adoption: llms.txt is a proposal, not a standard, and no major provider guarantees fetching it. As of mid-2026 adoption is real but uneven; several documentation platforms generate one automatically, many AI-adjacent companies publish one, and some agent tooling checks for it. It costs one static file to be early here, and nothing if you are wrong.

A complete example (adapt and ship)

This example is for a store; swap the sections for docs, features, or services as needed. Serve it at /llms.txt as plain text or markdown.

markdown

# Acme Outdoor Gear

> Acme sells lightweight backpacking gear with a 60-day return
> policy and free US shipping over $75. The links below are the
> current catalog, policies, and support pages.

## Products
- [Tents](https://acme.example/collections/tents): 1-3 person ultralight tents, $249 to $599
- [Sleeping bags](https://acme.example/collections/bags): down and synthetic, all seasons
- [Best sellers](https://acme.example/collections/best-sellers): top 20 by order volume

## Buying
- [Shipping and returns](https://acme.example/policies/shipping): rates, carriers, 60-day returns
- [Size guide](https://acme.example/pages/sizing): fit charts for apparel and packs

## Support
- [FAQ](https://acme.example/pages/faq): payment, warranty, and repair answers
- [Contact](https://acme.example/pages/contact): email and response hours

## Optional
- [Blog](https://acme.example/blog): trail guides and gear care articles

What makes one genuinely useful

Write each description as the answer an agent is looking for, not as marketing. Shipping and returns: rates, carriers, 60-day returns beats Learn more about our world-class service. Keep URLs absolute, keep the whole file under a couple thousand words, and link the pages that answer buying questions first: pricing, policies, categories, docs.

Treat it like code: regenerate it when your catalog or policies change, and keep it consistent with the pages it points to. A map that points at dead or stale pages is worse than the guessing it replaces.

Serving details are unfussy: a static file at /llms.txt returning 200 with a text content type is all any consumer expects. Pair it with a sitemap rather than replacing one; the sitemap enumerates everything for crawlers, while llms.txt curates what matters for a context window.

How AgentReady checks it

The scanner asks: is there a useful llms.txt file? A 404 fails. A file that exists and parses as markdown with at least one heading and at least three links passes; a file that exists but is thin warns. The check is worth 4 points in the Comprehension pillar.

The methodology's reasoning: an llms.txt file is the emerging map agents read first. A good one points them straight at the pages you want cited and bought from.

Frequently asked questions

Is llms.txt an official standard?: No. It is a community proposal documented at llmstxt.org. As of mid-2026 no major model provider commits to fetching it, but adoption keeps growing because the cost is one static file and the downside is zero.
How is llms.txt different from robots.txt?: robots.txt controls access: which paths crawlers may fetch. llms.txt aids comprehension: which pages are worth fetching and what each contains. One excludes, the other curates; sites benefit from both.
Do I also need llms-full.txt?: Optional. It inlines full content for agents that want everything in one fetch, which suits documentation sites. A store or SaaS marketing site gets most of the value from a well-curated llms.txt alone.