Bridge SEO
llms.txt: what the file declares, and who reads it
llms.txt is a plain markdown file at the root of a site that offers language models the site's own account of what it is and where its important pages are. It sits beside the tags covered under AI SEO. Advice to add one rarely says what the file claims or who has agreed to read it.
What the file is
llms.txt is a plain markdown file at the root of a site, proposed in September 2024 by Jeremy Howard of Answer.AI and specified at llmstxt.org. It offers a language model a short, curated account of what the site is and where its important pages are, written for a reader that arrives with a question rather than a browser.
The shape is deliberately small: an H1 with the site name, an optional blockquote summary, then lists of links with one line of description each. A companion file, llms-full.txt, carries the full text of those pages in one place, so a model can read the substance without following every link.
That is the whole proposal. It is a summary a site writes about itself, in a format a machine does not have to guess at.
What it is not
llms.txt is not a standard in the sense that a status code or a rel value is a standard. No standards body has ratified it, and no major engine has published a commitment to consume it. Treat any claim that a particular assistant reads your llms.txt as unverified unless that assistant says so in its own documentation.
It is also not an access-control file. It grants nothing and forbids nothing. A crawler that ignores it is not violating anything, and a crawler that reads it gains no permission it did not already have.
How it differs from robots.txt
robots.txt governs access: which paths a crawler may fetch. llms.txt offers content: what the site is about and where its important pages are. A site can have both, and they never contradict each other; the full comparison is under llms.txt vs robots.txt.
What it can plausibly do
The file can do three modest things, each narrow and each real.
It removes a guess
A model that would otherwise infer what a site is from whatever page it happened to land on gets a description the site wrote on purpose.
It shortens a path
Where the important pages are named in one place, a reader that follows links does not have to find them by crawling.
It survives your navigation
A menu is designed for people and often hides structure behind interaction. A flat list does not.
None of that is a ranking claim, and none of it is measurable in the way a redirect or a tag is. The file is a statement of intent, and its value depends entirely on whether the reader chose to read it.
The ceiling
llms.txt cannot make a page eligible for anything. A page that is not fetchable does not become fetchable because a summary mentions it.
Nor does it substitute for the tags a page is missing: a model reading a page directly reads that page's own head, and an llms.txt two directories up does not fill a missing description. Those gaps are the subject of the tag families, not of this file. And there is no console, no status and no confirmation that any reader used it, which is why claims about its effect deserve more scepticism than most.
Google has said so for its search engine directly. Its guide to generative AI features in Search, last updated on and checked on , lists llms.txt among things you can ignore: Google Search "doesn't use them," and maintaining one "will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them." The same guide adds that it is "completely fine" to keep one for other services or systems that use it.
Whether your site needs one
A file this small is rarely wrong to have, and the question is better asked as what it would say.
A site whose structure is genuinely hard to infer from its navigation gains the most: documentation with many entry points, a knowledge base whose hubs are not obvious, a product whose name does not describe it. A site of eight pages with a plain menu gains little, and a stale file that names pages you deleted is worse than none.
Writing it is a short editorial job, not an engineering one. The cost of keeping it accurate is the real cost, and it is the reason a file added once and forgotten stops being useful within a release or two.
What this site does with it
This site publishes its own llms.txt and llms-full.txt, and they are generated from the same sources as the pages, so they do not quietly drift from what the pages say.
Nothing on this site generates an llms.txt for your site. Enrichment fills missing tags in the copy served to machines and does not write files at your root. Where a page here describes what is shipped, that is the list, and this file is not on it.
FAQ
Does an llms.txt affect rankings?
Not in Google. Google's guide to generative AI search, updated on , says Google Search ignores llms.txt files and that keeping one "will neither harm nor help" a site's visibility or rankings. The file addresses readers that answer questions rather than rank pages, so keep it for those readers, not as a ranking lever.
Which assistants actually read it?
No major provider has published a commitment to read it. Some smaller tools and readers do, and say so in their own documentation. Any list claiming a large assistant consumes the file is worth checking against that assistant's documentation before you repeat it, because such lists circulate faster than the confirmations behind them.
Is it a replacement for robots.txt?
No, and the two do not overlap at any point. robots.txt governs access, has an established format, and is honoured by the major crawlers. llms.txt offers content, is a proposal rather than a format anyone has ratified, and is honoured by nobody in particular. Keep both if you want both, and expect neither to do the other's job.
Should llms-full.txt include every page?
It should include the pages you would hand somebody who asked what the site covers, which on a small site may well be all of them. A dump of everything is easy to generate and hard to read, and on a large site it defeats the point of writing a summary in the first place. The test is whether a reader could answer a real question from it.
How often should it be updated?
Whenever the pages it names change enough to make it wrong. A file that lists deleted pages misdescribes the site more confidently than no file at all, which is why generating it from the same source as the pages is worth the small effort.