Is llms.txt an official standard?

No. It is a community proposal published in 2024, not a ratified standard from a body such as the IETF or W3C. That may change, but today it has no formal status and no guaranteed support.

Will adding llms.txt get me cited by ChatGPT?

There is no evidence that it will. Citation depends on being retrievable and quotable, which comes from crawl access, page structure and third-party corroboration. Treat llms.txt as a small optional extra, not a mechanism.

Does llms.txt stop my content being used for AI training?

No. It has no access-control function whatsoever. Restricting training use is attempted through robots.txt directives aimed at specific crawler user agents, and compliance with those is voluntary on the crawler's part.

Where exactly should the file live?

At the root of the domain, at /llms.txt, served as plain text or Markdown rather than rendered as an HTML page. Subdirectory locations are not part of the proposal and should not be expected to work.

Should I also publish .md versions of my pages?

Only if you can generate them automatically from the same source as your HTML. Hand-maintained duplicates drift out of sync, and two contradictory versions of the same page is a worse outcome than one.

Can llms.txt cause any harm to my site?

Not directly, provided it does not expose URLs you intended to keep private. The realistic risk is opportunity cost — an hour spent on it instead of on rendering, structure or entity consistency, all of which have demonstrable effects.

The llms.txt File: What It Is and Whether You Need One

Key Takeaways

llms.txt is a proposed convention for publishing a curated, plain-text map of your most useful pages at the root of your domain, aimed at large language models rather than search crawlers.
It is a proposal, not a ratified standard, and no major AI provider has publicly documented consuming it — so treat any claim that it drives citations with scepticism.
It does not control access, does not block training, and does not replace robots.txt or sitemap.xml, all of which serve different and non-overlapping purposes.
The realistic upside is a cleaner ingestion path for anyone who does read it, at a cost of roughly an hour, which is why adding one is defensible even without proof it helps.
Anyone with a stale llms.txt is worse off than someone with none, because a contradicted file is a signal of neglect rather than of curation.

The llms.txt file is a proposal for a plain-text document at the root of your domain that gives large language models a clean, curated map of your site — the pages you would actually want an AI to read, in Markdown, without navigation, cookie banners or scripts in the way. It was proposed publicly in late 2024 and it has spread quickly through SEO circles. Whether it does anything for you is a more interesting question than most write-ups admit.

What is the llms.txt file?

An llms.txt file is a Markdown document served at https://yourdomain.com/llms.txt that lists your most important URLs with a one-line description of each, grouped into sections. It is intended to be read by a language model at inference time, when it needs to understand what a site contains without crawling and parsing every page.

The proposed format is deliberately simple: an H1 with the site name, a blockquote summarising what the site is, then H2 sections containing bulleted links with short descriptions. A companion convention suggests offering clean Markdown versions of pages at a .md suffix, so a model can fetch the content without the HTML shell.

How is it different from robots.txt and sitemap.xml?

They solve three different problems and none of them substitutes for another. robots.txt controls access, sitemap.xml declares completeness, and llms.txt proposes curation.

	robots.txt	sitemap.xml	llms.txt
Purpose	Permit or forbid crawling	List every URL worth indexing	Highlight the pages most worth reading
Format	Directive text	XML	Markdown
Audience	All crawlers, by user agent	Search engine crawlers	Language models
Status	Long-established, universally honoured	Long-established, universally supported	Community proposal, adoption unconfirmed
Enforces anything?	Yes, by convention	No	No
Effort to maintain	Low	Automated	Manual unless you generate it

The row that matters is the last-but-one. robots.txt is honoured because two decades of infrastructure agreed to honour it. llms.txt currently has no such agreement behind it.

Does anyone actually read llms.txt?

As of writing, no major AI provider has publicly documented that its systems read llms.txt, and you should be sceptical of anyone claiming otherwise without a link to provider documentation. Some developer tools and documentation platforms have adopted it, and it is genuinely useful in that context. For a general business website, the honest position is that adoption is unproven.

This is not a reason to sneer at it. Conventions have to start somewhere, and the cost of participating is close to nothing. It is a reason to reject the framing you will see in agency pitches, where llms.txt is presented as the lever that gets you cited. It is not. Being crawlable, being clear and being corroborated is what gets you cited, which is the argument laid out in generative engine optimization vs SEO.

Free consultation

Told you need an llms.txt file urgently?

We will look at your site and tell you honestly whether it is anywhere near your top ten priorities, and what is. No sales pressure and no obligation.

Get a free consultation

How do you write an llms.txt file if you want one?

Write it as a curated shortlist, not a second sitemap. The entire value of the format is that it is selective — a file listing 400 URLs defeats the purpose and is worse than not having one.

A workable structure for a services business:

# Example Studio > Digital, product and AI studio based in Gurugram. We build websites,
> apps and AI systems, and run SEO and performance marketing for
> Indian businesses. ## Services
- [SEO](https://example.in/seo-company): Technical SEO, content and local search.
- [Web development](https://example.in/web-development-company): Custom builds. ## Guides
- [Website cost in India](https://example.in/guides/website-development-cost-india): Real price bands and what moves them. ## Company
- [Contact](https://example.in/contact): Enquiries and consultations.

Rules that keep it useful:

Keep it under roughly 50 links. If you cannot choose, you have not curated.
Write descriptions that state what the page contains, not what you want the reader to feel.
Use absolute URLs.
Serve it as text/plain or text/markdown, not as an HTML page.
Regenerate it whenever your site structure changes, and put that step in your deploy checklist.

What does llms.txt not do?

It does not control anything. This is the most consequential misunderstanding, and it leads businesses to think they have protected their content when they have not.

It does not block training. Only robots.txt directives against the relevant user agents attempt that, and even then compliance is voluntary.
It does not grant access. If robots.txt disallows a path, listing it in llms.txt changes nothing.
It does not affect Google rankings. Google has not indicated any use of the file.
It does not make a page quotable. A badly written page linked from llms.txt is still a badly written page.
It does not replace a sitemap. Search engines still need the complete XML list.

If your actual goal is controlling how AI systems interact with your content, the file you need to edit is robots.txt, and the decisions you need to make are about specific user agents. That is a genuinely useful hour of work, and it is one we walk clients through as part of any AI engagement where content and data policy come up.

So should you add one?

Add one if your site is small enough to curate in under an hour and you have someone who will keep it current; skip it if it will be published once and abandoned. A stale llms.txt pointing at pages that have moved is a worse signal than no file at all, because it advertises that nobody is maintaining the site.

Rank it honestly against your other options. Fixing rendering so your content exists without JavaScript matters more. Restructuring your top pages into question-and-answer format matters more. Getting your Organization schema and profiles consistent matters more, as covered in schema markup for AI search visibility. On a large site, sorting out how crawlers spend their time matters more too, which is the subject of crawl budget optimisation for large sites.

Once those are done, spend the hour. It is cheap, it is reversible, and if adoption does arrive you will already be there. Just do not let it be the thing you did instead of the work that actually moves visibility — that ordering is the whole point of how we sequence technical SEO engagements, and there is more background reading in our guides library.