llms.txt explained: what it is and whether your business needs one
llms.txt is a proposed file that gives AI agents a map of your site. Who proposed it, what goes in it, what Google says about it and our honest recommendation.
llms.txt is a plain-text file at the root of a website, for example
yoursite.com/llms.txt, that gives AI agents a short summary of the site and
a list of links to its most useful pages. It is a sensible idea, it costs very
little to add, and it will not, by itself, make any AI assistant recommend your
business.
That last part is where most of the confusion sits, so this piece separates what is documented from what is hoped for.
Where it came from
The proposal was published by Jeremy Howard on September 3, 2024, at
llmstxt.org, with a second version in August 2026. Its
stated aim is “to standardise on using an /llms.txt file to provide
information to help agents use a website”, giving “brief background
information, guidance, and links to detailed markdown files”.
The thinking behind it is reasonable. Web pages carry navigation, scripts, cookie banners and layout that an AI agent has to wade through. A short file written for agents, pointing at clean versions of the important pages, saves them that work. The author expected it to be used mainly when an agent is answering a question, rather than for training.
What goes in the file
The format is ordinary markdown, in this order:
- An H1 with the site or project name. The only required part.
- A blockquote with a one-paragraph summary.
- Optional paragraphs with more detail.
- H2 sections listing links, each written as
[name](url): notes. - An “Optional” H2 for secondary links an agent can skip.
For a local service business, it might look like this:
# Summit Plumbing
> Licensed residential plumber serving Omaha, NE. Emergency repairs,
> water heaters and repiping. Mon to Sat, 7am to 7pm.
## Services
- [Emergency plumbing](https://example.com/emergency.html): 24/7 call-outs, typical arrival times
- [Water heaters](https://example.com/water-heaters.html): repair vs replace, price ranges
## About
- [Service area](https://example.com/areas.html): neighborhoods and towns covered
- [Reviews](https://example.com/reviews.html): verified customer reviews
## Optional
- [Blog](https://example.com/blog.html)The proposal also suggests offering clean markdown copies of key pages at the
same address plus .md, so an agent can read the content without the page
furniture.
Who actually uses it
This is where it pays to be precise.
Publishing it is common. The llms.txt site reports that thousands of sites publish one, and that OpenAI, Anthropic and Google’s Gemini team publish llms.txt files for their own developer documentation.
Reading it to choose sources is not documented. We could not find documentation from OpenAI, Google, Perplexity or Anthropic stating that their consumer assistants use a site’s llms.txt to decide which businesses to recommend or which pages to cite. Publishing a file for your developer docs is not the same as using everyone else’s file to rank answers.
Google says you do not need it for AI search. Its guidance on AI Overviews and AI Mode states: “You don’t need to create new machine readable files, AI text files, or markup” (Google Search Central).
Where llms.txt is most useful today is with agents and developer tools, such as an assistant reading software documentation, or an agent asked to go and read a specific site. That is a real use, just not the one most marketing claims describe.
Our recommendation
Add one if you like, after the fundamentals, and do not pay much for it.
| Priority | Work | Why |
|---|---|---|
| First | Let AI crawlers in (OAI-SearchBot, PerplexityBot, Googlebot) |
Documented requirement for appearing at all |
| First | Make key pages answer questions plainly and specifically | What gets quoted |
| First | Consistent business details across the web | What gets you trusted |
| Later | Publish an llms.txt | Cheap, harmless, helps agents navigate |
A small business can write one in under an hour. If someone quotes you a significant fee for “llms.txt optimization”, ask them which assistant they can show reading it to choose recommendations. We publish our own, because it costs nothing and some agents can use these files, not because it moves rankings.
Questions people ask
Will an llms.txt file get me into ChatGPT answers?
Not on its own. OpenAI documents crawler access through OAI-SearchBot as what
determines whether a site can appear in ChatGPT search. We found no OpenAI
documentation saying it reads llms.txt for that purpose.
Is llms.txt an official standard? No. It is a community proposal maintained at llmstxt.org, not a standard from a standards body or a search engine.
Can llms.txt hurt my site? Not if it is accurate. It is a public file, so do not list anything you would not want an agent or a person to read.
Is llms.txt like robots.txt? No. Robots.txt tells crawlers what they may access. llms.txt only describes and links content; it grants or blocks nothing.