llms.txt has gone from obscure proposal to agency upsell in under two years. Here is what it actually is, what the evidence says it does, and where it fits among the things that do move AI visibility.
What llms.txt is
The llms.txt proposal, published by Jeremy Howard
on 3 September 2024, suggests adding a file at /llms.txt that gives AI models
"concise, expert-level information gathered in a single, accessible location" — without the
navigation, scripts and adverts they would otherwise have to wade through on an HTML page.
The format is ordinary Markdown, in a fixed order:
- An H1 with the name of the site or project — the only required part.
- A blockquote summarising it in a sentence or two.
- Optional paragraphs or lists with more detail.
- Optional H2 sections of links to the most important pages, each with a short note.
- By convention, a section called Optional for links a model can skip when short of space.
The proposal also suggests publishing clean Markdown versions of individual pages, and some sites
add an llms-full.txt with their documentation concatenated into one file.
What it does — and does not — do
It is not a ranking factor in Google
Google is explicit. Its guidance on AI features says: "You don't need to create new machine readable files, AI text files, or markup to appear in these features." AI Overviews and AI Mode run on the same foundations as the rest of Google Search.
No assistant has confirmed it reads yours
Several AI companies publish llms.txt files on their own developer documentation. That is not the same as their crawlers reading the file on your site and using it to decide what to say about you, and none has publicly confirmed that they do.
Where it clearly helps
Tools that are pointed at a site on purpose — a coding assistant reading a library's docs, an agent asked to research a company — can fetch one compact file instead of crawling dozens of pages. That is the use it was designed for, and it is where the benefit is real today.
Should you publish one?
Usually yes, because it is cheap — Google says you do not need one, not that you should avoid one. Two rules make it worth the half hour:
- Only put confirmed facts in it. A model that reads the file may repeat it. Opening hours you have not confirmed, prices you would not stand behind, a service area you do not cover — leave them out rather than guess.
- Keep it consistent with the site. If the file says one thing and your pages say another, you have made the facts about your business less reliable, not more.
This site publishes its own at /llms.txt. It deliberately leaves out hours and pricing, because neither is settled.
What actually moves AI visibility
If llms.txt is the cheap extra, these are the foundations. An assistant cannot mention what it cannot reach, read or trust:
- Crawler access. A robots.txt that does not block the assistants' crawlers, and a site that actually loads for them.
- Pages that render without JavaScript. Many AI crawlers do not run scripts, so content assembled in the browser can be invisible to them.
- Answers written to be quoted. A self-contained paragraph that names the subject is liftable; one that relies on the sentence before it is not.
- Structured data that matches the page. Schema.org markup for your organisation, services and FAQs, describing only what a visitor can see.
- Consistent facts everywhere. The same name, address and phone number on your site, your Business Profile and every directory.
We cover those in more depth in how to get your business mentioned by AI assistants.
Sources
- The /llms.txt file — Jeremy Howard, llmstxt.org, proposed 3 September 2024
- AI features and your website — Google Search Central