LLMs.txt: A Year On, Is It Actually Working?

Here is what actually happened in the twelve months after the LLMs.txt proposal landed.

Eight hundred and forty-four thousand websites added the file. Anthropic implemented it on their own documentation. So did Cloudflare. So did Stripe. Google briefly rolled it out across their developer properties in December, then quietly pulled it back within a day — with John Mueller clarifying it had gone live by accident via a CMS update rather than by intention.

And the major AI platforms (OpenAI, Anthropic, Google, Microsoft) have not publicly committed to reading or acting on the files in their production systems. Not one.

That is the state of LLMs.txt at roughly eighteen months old.

I wrote about the proposal in October 2024 when Jeremy Howard at Answer.AI first put it forward. The idea is simple: a plain-text file at /llms.txt that tells AI language model crawlers what your website is, what matters, and what you would like them to know. Markdown. Curated links. A map for the machine rather than the human. The robots.txt analogy is right: not a protocol, not a W3C standard, just a de facto convention that spreads because the logic is sound.

By October 2025, around 844,000 websites had implemented some form of LLMs.txt, according to BuiltWith tracking. For a voluntary convention with no formal standards body behind it, that rate of uptake is genuinely impressive. Fourteen months, no official platform mandate, and nearly a million sites adopted it anyway.

And yet.

The reading problem

Implementing LLMs.txt and having AI crawlers actually read and act on it are two different things.

OpenAI's GPTBot fetches the file occasionally. In some site logs, it turns up every fifteen minutes. That is crawling, not acting on. There is no clear mechanism by which a site's LLMs.txt content is preferentially incorporated into the training or retrieval architecture that powers the AI answers that matter to ecommerce businesses.

A Search Engine Land study published in January 2026 tracked ten sites across 90 days before and after implementation, covering finance, B2B SaaS, ecommerce, insurance, and pet care. Eight saw no measurable change in AI-referred traffic. Two saw gains (12.5% and 25%) but the researchers traced those to content restructuring and fixing crawl errors, not the LLMs.txt file itself.

Google's John Mueller has said plainly that no AI system currently uses LLMs.txt. The Google team has explicitly stated they have no plans to support it.

The distinction matters. If you are optimising for AI discoverability, the thing that matters is what gets retrieved and cited, not what gets crawled. A crawled-but-not-cited LLMs.txt is commercially equivalent to a beautifully formatted robots.txt that no one reads.

The honest current state: there is no clean correlation between LLMs.txt implementation and improved AI citation rates.

What it is actually good for

I want to resist writing the "LLMs.txt is overhyped" piece, because I think that is also slightly wrong.

Two things LLMs.txt does reasonably well right now, independent of whether the major crawlers formalise support.

The first is RAG. If you are building or commissioning a retrieval-augmented generation product (a customer service chatbot, an internal knowledge tool, anything where your own content is being chunked and indexed), LLMs.txt is a sensible way to structure what matters and what does not. It is a gift to whatever RAG pipeline is ingesting your site. The convention makes sense even without broad crawler support because it solves a real problem when you control both ends.

The second is declarative intent value. A site with a well-maintained LLMs.txt is a small signal that you have thought about how AI systems should interact with your content. Whether that signal propagates to AI citation behaviour right now is uncertain. Whether it matters over a two-to-three year horizon, as AI crawlers mature and the infrastructure consolidates, is a different question.

The pragmatic argument from October 2024 still holds: the cost is low enough that the option value justifies implementation. If anything, it has strengthened. The sites that have it now will have a documented implementation history when and if the major crawlers formalise support.

The ecommerce priority order

For product-first ecommerce businesses, the honest priority order has not changed much from a year ago.

Schema.org Product markup remains the single investment with the clearest evidence of current AI use. Google confirmed structured data gives a ranking advantage in April 2025. Fabrice Canel at Microsoft Bing confirmed that schema markup helps Bing's LLMs understand content for Copilot. I covered this in January when the confirmations came through.

LLMs.txt belongs in the second tier: worth doing, particularly for retailers with a substantial editorial layer (buying guides, category content, expert recommendations) that does not naturally surface through product structured data. An LLMs.txt that surfaces your size guides and sustainability credentials gives an AI useful context when asked "where should I buy a good winter coat?", even if no systematic crawler processes it today.

AI discovery infrastructure is still being built. LLMs.txt, structured data, GEO-optimised content, better machine readability: these are preparatory investments in a landscape that will look substantially different in eighteen months.

That is not a reason to skip them. It is a reason to be clear-eyed about what they are doing right now versus what we are betting they will do later.

The 844,000 implementations are, in aggregate, a signal that the industry is taking this seriously. Whether the AI platforms will meet that seriousness with the infrastructure to make it meaningful is, as of February 2026, still open.

I will revisit in another year.