// Answer

What is llms.txt and do I need one?

llms.txt is a plain-text file at your site root that tells AI crawlers which pages and content are most useful to read. Here is what it does, what goes in it, and whether you need one.

The short answer

llms.txt is a plain-text file published at your site root (yoursite.com/llms.txt) that points AI crawlers and answer engines to the pages and documents you most want them to read and cite. It works like a curated map of your best content, written in simple Markdown, rather than a blocking rule. You do not strictly need one to be cited, but it helps large sites surface their canonical pages, reduces guesswork for engines, and complements (not replaces) robots.txt and your sitemap.

What goes in the file

An llms.txt file is short and human-readable. It usually opens with an H1 of your site or brand name, a one-line summary of what you do, then a few sections of links. Each link is typically a title, the URL, and a brief note on what the page covers. The goal is to surface your genuinely useful pages: documentation, canonical explainers, key product pages, and reference material. Leave out thin, duplicate, or low-value URLs. A tight file of twenty strong pages is more useful to an engine than a dump of every URL you have.

Do you actually need one

You can be cited without llms.txt, because engines still crawl and retrieve normally. It helps most when your site is large, your best content is buried, or your information architecture makes the important pages hard to find. Treat it as one layer: robots.txt handles permission, your sitemap is the full index, and llms.txt is the curated short list on top.

How it fits the bigger picture

llms.txt only pays off if crawlers can actually reach your pages and extract clear answers from them. See the AI citation tracking methodology for how access and extractability feed into citations, or the what is AEO primer for the full practice. To check whether your site exposes llms.txt and lets AI crawlers in, run the free AI visibility scan.

Frequently asked questions

Is llms.txt the same as robots.txt?

No. robots.txt controls which crawlers are allowed to access which paths (permission), while llms.txt is a curated guide to the content you most want AI engines to read and cite (recommendation). They serve different jobs and can live side by side: robots.txt decides access, your sitemap lists everything, and llms.txt highlights the handful of pages that best answer questions in your category.

Where do I put llms.txt and what format is it?

Place it at your site root so it resolves at yoursite.com/llms.txt, served as plain text. The format is simple Markdown: a top heading with your site name, a short description, then sections of links (often a title, a URL, and a one-line note per link). Keep it to your genuinely useful pages, like docs, key product pages, and canonical explainers, rather than your entire URL list.

Run your free scan in 60 seconds

Run a free scan to see where you stand across ChatGPT, Claude, Gemini, and Perplexity: which answers cite you, which cite competitors instead, and what to fix first.

Run a free scan