Does llms.txt do anything?
We generate llms.txt files and charge for them, so read this with that in mind. The honest answer is that no major assistant is documented as fetching it, and it is still worth twenty minutes.
·3 min read
Figures as of 9 September 2026, taken over 350 scanned sites. They move as scans arrive — the live version is here.
Forty-two per cent of the sites we scan have no llms.txt. We check for it, we generate one in the fix pack, and we charge money for that fix pack. So here is the conflict of interest up front, and then the actual answer.
What it is
A markdown file at /llms.txt: what this site is, and a list of its most useful pages with a sentence about each. Optionally /llms-full.txt, which inlines the whole text of those pages so a client needs one request instead of seven. The proposal came from Jeremy Howard in 2024. It is a convention, not a standard — there is no working group and no specification with a version number.
The honest state of adoption
No major assistant publicly documents fetching llms.txt as part of answering a question. Not OpenAI, not Anthropic, not Perplexity, not Google. People will tell you otherwise. Ask them for the documentation.
What is true is that a growing number of tools do read it: documentation platforms, coding agents pointed at a library, a range of smaller crawlers, and anything a developer builds against a site deliberately. That is a real audience. It is just a different and smaller one than the pitch usually implies.
Why we still say write one
Three reasons, none of them "assistants read it".
- It costs twenty minutes and it cannot hurt. The downside case for a small static file is that nothing fetches it.
- Writing it is the useful part. Naming your ten most important pages and saying in one sentence what each is for is an exercise most sites have never done, and it surfaces the pages nobody can describe.
- It is cheap insurance on a convention that might get adopted. If it does, you already have one. If it does not, you spent twenty minutes.
The thing that actually works today
Serving the same page as markdown when a client asks for it. Seventy-seven per cent of sites have no plain-text or markdown alternative at all, which is a much bigger gap than the llms.txt one and it has a mechanism that already exists: content negotiation, which every HTTP client understands.
GET /pricing HTTP/1.1
Accept: text/markdown
HTTP/1.1 200 OK
Content-Type: text/markdown; charset=utf-8
Link: </pricing.md>; rel="alternate"; type="text/markdown"A client that wants your words should not have to run a browser and then strip tags to get them. This site does it on every public page, including the one you are reading — add .md to the URL, or send the header. Our API and docs page lists everything machine-readable we serve.
What a good one looks like
Short. A title, one line saying what the site is, then links with a sentence each. Not a sitemap — a sitemap is every URL, and this is the ten that matter. Not marketing copy. The test is whether somebody who has never heard of you could answer a question about you from it.
If you want ours as a shape to copy it is at /llms.txt, generated from the same page list that builds the sitemap so the two cannot disagree. And if you want one built from your own site, from the URLs a scan confirmed actually return 200, that is what the fix pack is.