If you have been reading about AI search, someone has probably told you that your website needs an llms.txt file. So what is llms.txt, and should a small business bother? The short version: it is a simple, sensible idea that costs very little, but it is not the secret to appearing in ChatGPT or Google's AI answers, and anyone selling it that way is overselling.
Here is what the file is, where it came from, what goes in it, and an honest view of how much it matters.
Where llms.txt came from
llms.txt was proposed in September 2024 by Jeremy Howard, co-founder of Answer.AI, and is documented at llmstxt.org. The idea grew out of a practical problem. Large language models have limited room to read in one go, and a normal web page is full of menus, cookie banners, scripts and layout code. A model trying to understand a website has to wade through all of that to find the useful bits.
The proposal is a single Markdown file at the root of a website, in the same spirit as robots.txt and sitemap.xml, that says in a few hundred words what the site is and where the important information lives. Markdown was chosen because it is readable by people and easy for models to process.
It is a proposal, not a formal standard agreed by the companies that run AI assistants. That distinction matters for everything that follows.
What goes in an llms.txt file
The proposed format is simple. In order:
- A heading with the name of the business or site (the only required part).
- A short summary in a blockquote: one or two sentences saying what you are.
- Optional paragraphs of key facts that do not fit elsewhere.
- Sections headed with “##”, each containing a list of links to important pages with a short note on what each contains.
- An optional section titled “Optional” for links a tool can skip if it is short of space.
An example for a small business
For a local business, a useful llms.txt is short and factual. A dental practice might write something like this, in plain Markdown:
- # Riverside Dental, Taunton
- > Independent NHS and private dental practice in Taunton, Somerset, accepting new private patients and a limited number of NHS patients.
- Key facts: open Monday to Friday 8am–6pm and Saturday mornings; emergency appointments held back daily; wheelchair access; registered with the Care Quality Commission.
- ## Services: links to the pages for check-ups, hygiene, implants, Invisalign and emergency care, each with one line.
- ## Practical: links to fees, finding us, new patient registration and contact.
Who actually uses llms.txt?
This is where honesty matters. At the time of writing, none of the big consumer assistants has publicly committed to reading llms.txt when deciding what to say about a business or which business to recommend.
Google's guidance on AI Overviews and AI Mode says no special files or markup are needed beyond normal indexing, and llms.txt does not appear in it. OpenAI, Anthropic and Perplexity document their crawlers and how robots.txt controls them, which is the mechanism they do describe publicly. None of them lists llms.txt as a ranking or recommendation signal.
Where llms.txt is genuinely used is in developer tools. Many software documentation sites publish one so that coding assistants and AI agents can load the docs efficiently, and some tools will read a site's llms.txt if a user points them at it. An AI agent browsing on someone's behalf might also find it useful. That is real, but it is not the same as being recommended to customers.
| Claim | Reality |
|---|---|
| “It gets you into ChatGPT answers” | Not documented by OpenAI; crawler access and good pages matter far more |
| “Google uses it for AI Overviews” | Google says no special files are needed; llms.txt is not in its guidance |
| “It controls AI training on your content” | No. That is what robots.txt rules for specific crawlers are for |
| “AI tools can read your site more easily” | True for tools that choose to read it, mostly developer and agent tools |
| “It is costly to set up” | No. For a small site it is one short file, maintained when your key facts change |
Should your business have one?
If your website is well built, adding an llms.txt file is a small, low-risk extra. It takes little time, it cannot hurt your search visibility, and if assistants do start relying on it, you will already have one. It also forces a useful exercise: writing down, in a few lines, exactly what your business does and which pages matter. Many websites could do with that discipline on the home page too.
What it should never be is a priority over the basics. If AI crawlers are blocked by your hosting, if your pages do not say clearly what you do and where, or if your details are inconsistent across the web, an llms.txt file will not rescue you. Fix those first.
And if someone quotes you a significant fee for “llms.txt optimisation”, decline. It is a text file.
Common llms.txt mistakes
- Writing it as an advert. Superlatives and slogans give a model nothing to verify and read as noise.
- Listing facts that contradict the website, usually because the file was written once and forgotten when prices or hours changed.
- Pasting every page of the site into it. The point is a short guide; the full text belongs in the pages themselves or an optional llms-full.txt.
- Treating it as a replacement for robots.txt. It does not allow or block any crawler.
- Assuming it fixes blocked crawlers. If an AI crawler cannot reach your site, it cannot read llms.txt either.
How to add one
- Write it in any text editor using the structure above. Keep it under a page or two for a small business.
- Stick to facts: services, areas, hours, contact routes, qualifications. No slogans.
- Upload it to the root of your site so it loads at yourdomain.com/llms.txt. On hosted builders this may need a developer or may not be possible yet.
- Make sure it matches your website, Google Business Profile and other listings exactly. Conflicting facts are worse than none.
- Update it whenever your prices, hours, services or contact details change, the same day you change the website.