llms.txt: what it is and whether your site needs one

For months now, SEO newsletters have kept running the same headline: «Add llms.txt or AI will not see you». It sounds urgent. The reality is calmer - and more interesting.

What it is

llms.txt is an ordinary text file at the root of a site, at /llms.txt. Briefly, in plain language and in Markdown, it describes what the site is, who is behind it and which pages carry the substance. The idea was proposed by Jeremy Howard in autumn 2024.

The logic is simple. An ordinary website is awkward for a language model to read: menus, footers, banners, scripts - more noise than substance. llms.txt hands it a cheat sheet: here is what matters, here is where to look.

What it is not

It is not robots.txt. It does not block or allow anything - crawler access to a site is still governed by robots.txt. And it is not a standard: no major search engine has committed to reading it. Google's representatives have said publicly that Google Search does not use it.

So if someone promises you that llms.txt will lift you in Google, that is not true. It has no effect on rankings in ordinary search.

llms.txt is not an «appear in ChatGPT» button. It is a business card - some will read it, some will not.

Then why bother

Because it costs half an hour, and search engines are not the only readers. Assistants someone has handed a link to your site, agents gathering information for an answer, developer tools - some of them look for this file. And when they find it, they describe you more accurately.

For a small site with a blog, it is cheap insurance against an AI describing your company from the footer text or from a page that went out of date long ago.

What to put in it

• A sentence or two on who you are and what you do - the way you would want to be retold
• Links to the main pages, with a short description of each
• Contact details
• Caveats, where they matter: for instance, that the prices on the site are starting rates, not final ones

And the one rule that matters: nothing that is not on the site itself. A file promising more than the pages back up is worse than no file - the assistant repeats the promise, and the visitor arrives to find it is not there.

How we do it

Our llms.txt is not a separate file somebody has to remember to update. It is generated from the same data as the blog and the sitemap: a new article goes out and appears in the file by itself. A small thing, but small things like that are what separate a file that helps from one that is lying six months later.

Do you need one? If people might ask an AI about your site - yes, it is the cheapest thing you can do. But if the site lacks proper HTML, markup and clear pages, llms.txt will not make up for it. Foundation first, business card second.