llms.txt: what it is, and whether it does anything yet
The honest status of llms.txt, what no AI company has confirmed about it, and the three things that demonstrably do decide whether an AI assistant can read your site.
the short answer: llms.txt is a proposal, not a standard. no major AI company has confirmed using it for retrieval or ranking. this site has one anyway, because it costs an afternoon and the downside is zero, but if you are hoping it is the reason you start getting cited, it is not.
What it actually is
a plain markdown file at the root of your domain, /llms.txt, holding a curated summary of the site: who you are, the key facts, and a linked map of the pages that matter, each with a line explaining what it covers.
the idea is reasonable. an AI assistant landing on a site has to work out the structure from navigation and markup built for humans. a hand written map removes the guesswork.
What nobody can honestly claim about it
that it improves your chances of being cited.
OpenAI, Anthropic, Google and Perplexity have not published anything confirming llms.txt is read as a retrieval or ranking input. absence of a confirmation is not proof it does nothing, but it does mean anyone selling you llms.txt as an AI ranking factor is selling you a guess with a confident voice.
i put one on this site on the day it launched. i cannot show you a single citation that traces back to it, and i am not going to pretend otherwise.
What demonstrably does decide whether an AI can read you
three things, all boring, all verifiable.
static HTML. AI crawlers do not execute javascript. a page that assembles itself in the browser is, to most of them, an empty page. this is the single biggest one and it is invisible in a browser, because your browser runs the javascript that the crawler does not.
robots.txt actually allowing them. GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot and Google-Extended are separate user agents with separate permissions. a site can rank fine in Google search and be blocked from every AI assistant at the same time, and most owners have never looked.
being indexed at all. a domain nobody has crawled cannot be retrieved by anything. i learned this one the expensive way: this site sat at zero indexed pages for its first week while i optimised things that could not matter yet.
The order that makes sense
check robots.txt is not blocking the AI crawlers. confirm your pages are static HTML and actually indexed. write clear answers to real questions, because retrieval is matching text to a question and there is no substitute for having the answer on the page.
then, if you like, add llms.txt. it is genuinely useful as a forcing function: writing a one line description of every page on your site tends to reveal how many of them do not deserve one.
the llms.txt generator will build the file from your site structure. it takes about a minute, which is roughly the right amount of effort to spend on something this speculative.
the wider version of how AI assistants actually pick what to recommend is in the AI search playbook.
want the honest read on whether AI assistants can even see your site? the breakdown call is free. Let’s talk Growth.