What is llms.txt, and does your website need one?
llms.txt is a proposed file that summarises your site for AI systems. Here is what it is, what is proven and unproven, and how to decide whether to add one.
If you work on search visibility, you have probably seen advice to add an llms.txt file. It sounds like robots.txt for AI. It is related, but it is not the same thing, and the claims around it often outrun the evidence. Here is what it is, what we actually know and how to decide.
What llms.txt is
llms.txt is a proposal for a plain-text file in Markdown at the root of a website, at /llms.txt. It gives a language model a short description of the site and a list of links to the most useful pages, with a sentence on each. The idea is that a model reading your site at the moment someone asks a question can find a clean, curated summary instead of parsing navigation, scripts and layout.
A longer companion file, often called llms-full.txt, can hold the full text of the site in one place.
How it differs from robots.txt and sitemaps
- robots.txt tells crawlers which parts of the site they may or may not fetch.
- sitemap.xml lists the URLs you want search engines to discover.
- llms.txt offers a human-readable summary meant to help a model understand what the site is and where its best content lives.
It does not block or allow anything, and it does not replace the other two.
What is proven, and what is not
Be honest about the state of evidence.
- It is a community proposal, not an official web standard.
- Search engines have publicly said they do not use it as a ranking signal.
- There is no solid evidence that adding one raises rankings or makes an AI assistant more likely to cite you.
- Some tools and assistants may read it, and some developers use it for documentation, but support is uneven and changes over time.
Anyone promising better AI visibility from this file alone is overselling it.
What does matter more
AI assistants and search engines mostly rely on the same fundamentals.
- Content that answers specific questions clearly and accurately, with the answer near the top of the page.
- A clear structure: descriptive headings, short sections, lists and tables.
- Visible authorship and expertise, with a real person or organisation behind the page.
- Structured data that matches what is on the page, such as Article, FAQPage, Service and BreadcrumbList.
- Pages that are crawlable, fast and well linked together.
- Mentions of you on other credible sites and profiles.
A clean llms.txt sits on top of these, it does not replace them.
Why add one anyway
There are reasonable arguments for it.
- It costs very little to produce and maintain.
- Writing it forces you to state plainly who you are, what you offer and which pages matter most, which is useful discipline.
- If an assistant or tool does read it, it gives a short, accurate source to quote.
The risk is staleness. A file that lists pages that no longer exist, or describes services you no longer offer, is worse than none.
How to write a good one
- Start with a one-paragraph description of the site and who it is for.
- State key facts plainly: the name, location, services, languages and how to contact.
- Group links by topic and add one factual sentence to each.
- Link to your most important pages only, not every URL.
- Keep it accurate, and update it when the site changes. Add the date you last updated it.
How I use it
This site publishes an llms.txt and an llms-full.txt, generated from the same content as the pages so they stay consistent. I treat them as a courtesy to any tool that reads them and not as a reason my pages should rank or be cited. The pages themselves, their structure and the expertise behind them do the real work. I also explain the wider picture in my article on what GEO is and how to make a website visible in AI search.
The takeaway
Add an llms.txt if you like, keep it honest and short, and spend most of your effort on content, structure and credibility. If you want a review of how your site reads to search engines and AI assistants, describe it and I can point out what is worth fixing first.
Frequently asked questions
What is llms.txt?
It is a proposed plain-text file, written in Markdown, placed at the root of a website to give language models a short, clean summary of the site and links to its most useful pages.
Does llms.txt improve rankings or AI citations?
There is no reliable evidence that it does, and search engines have said they do not use it for ranking. Treat it as a low-cost summary, not as a ranking factor.
How is llms.txt different from robots.txt and sitemap.xml?
robots.txt tells crawlers what they may access, sitemap.xml lists the URLs you want discovered and llms.txt offers a human-readable summary of the site's content. They do different jobs.
Should I add an llms.txt file?
It is cheap and harmless if kept accurate, and it forces you to write a clear description of your site. But do not expect it to replace good content, clear page structure and structured data.