llms.txt: Format, Examples, and llms-full.txt

llms.txt is a proposed Markdown file that gives AI agents an overview of a site and links to useful resources. llms-full.txt provides collected page content, serving a different purpose from an index. Publishing either file does not guarantee that an AI will fetch it automatically or that search rankings will improve.

How to write llms.txt

The official proposal orders its elements as a site-name H1, a blockquote summary, additional context, and link lists grouped under H2 headings. Only the H1 is required. Add brief link descriptions and, where possible, point to readable Markdown rather than HTML.

Files can live at /llms.txt or a path such as /docs/llms.txt. In v2, a file covers URLs beneath its path; the most specific applicable file takes precedence. The change notes explain that Optional remains a convention for secondary links, rather than a special instruction that tools must mechanically exclude.

When curating an index, start with an introduction, API reference, and limitations. Explaining which resource answers which question matters more than listing every URL.

An example using real public documentation

This minimal example uses real genko.me documentation URLs. It illustrates a file you could write; it is not a reproduction of the entire live index.

# genko.me docs
> Public documentation for integrating genko.me.
## SDK
- [Client](https://docs.genko.me/sdk/client.md): Configure the client.
## Optional
- [Full text](https://docs.genko.me/llms-full.txt): Read all documentation.

The live Japanese index contains Markdown page links and descriptions, followed by a pointer to the full-text file. An English index is also available. The documentation guide limits these resources to public documentation. They do not expose workspace articles or API keys. The Japanese documentation uses URLs without /en/; the English documentation uses that prefix.

How llms-full.txt differs

llms-full.txt is not part of the current proposal, and its filename alone does not define its scope or structure. genko.me supplies every documentation page's body, as described in its guide. A file with that name on another site may cover something different.

Prefer the index and individual pages for routine use. For a full-text file, consider input limits, transfer size, and irrelevant material. If it exceeds the tool's input limit, retrieve selected sections instead. A downloaded copy must also be refreshed when the source changes.

Generate the files from CMS or documentation content

Separating the CMS adapter from file generation makes the process easier to maintain. Normalize public content into a shared format, then generate outputs. The example below turns existing Markdown into static files; it does not fetch a vendor API or convert HTML.

Normalize the public input

Fetch every page of CMS results and select only versions intended for publication. If the body is HTML or block JSON, provide a separate conversion step that preserves headings, links, and code. Convert MDX components into readable Markdown rather than passing component syntax through unchanged.

Here is an example cms-public.json for a fictional site. published is a visibility flag defined by your adapter, not a claim that every CMS has a field with that name.

[  {    "slug": "guide",    "title": "Guide",    "summary": "Setup instructions",    "markdown": "## Setup\n\nInstall the SDK.",    "published": true  }]

Produce the index, pages, and full text together

Save this as generate-llms.mts and run node generate-llms.mts with Node.js 24. Execution strips type annotations; that command does not type-check the program. See Node.js TypeScript support. Replace origin with your public site's origin.

import { mkdir, readFile, writeFile } from "node:fs/promises";
type Page = {  slug: string;  title: string;  summary: string;  markdown: string;  published: boolean;};const origin = "https://example.com";const rows = JSON.parse(await readFile("cms-public.json", "utf8")) as Page[];const pages = rows.filter((p) => p.published === true);const seen = new Set<string>();for (const p of pages) {  if (!/^[a-z0-9]+(?:-[a-z0-9]+)*$/.test(p.slug) || seen.has(p.slug)) {    throw new Error("Invalid or duplicate slug");  }  seen.add(p.slug);}const line = (s: string) => s.replace(/[\r\n]+/g, " ");const label = (s: string) => line(s).replace(/[\\[\]]/g, "\\$&");const url = (p: Page) => `${origin}/docs/${p.slug}.md`;const body = (p: Page) =>  `# ${line(p.title)}\n\nSource: ${url(p)}\n\n${p.markdown.trim()}\n`;const index = [  "# Example docs", "", "> Public integration documentation.", "", "## Docs", "",  ...pages.map((p) => `- [${label(p.title)}](${url(p)}): ${line(p.summary)}`),  "", "## Optional", "",  `- [Full text](${origin}/llms-full.txt): All selected pages.`, "",].join("\n");await mkdir("public/docs", { recursive: true });for (const p of pages) await writeFile(`public/docs/${p.slug}.md`, body(p));await writeFile("public/llms.txt", index);await writeFile("public/llms-full.txt", pages.map(body).join("\n"));

This short example assumes the JSON shape has already been validated. For production, add schema validation, abort generation if API fetching fails, and check the article count. Filtering a flag is insufficient unless the fetched body itself is the public version.

Generate into a clean build directory. This code does not delete previous files: withdrawing a previously published page also requires removing its old .md output from deployment.

Next.js can serve static files from its public folder. Write this code's output to public and run the generation step before the build. CMS updates should refresh the HTML, index, individual bodies, and full text together. Next.js webhooks and revalidation discusses update paths; dynamic revalidation alone does not rewrite these static files.

How robots.txt and sitemap.xml differ

Google's robots.txt guide explains that crawl control does not secure private content, and a blocked URL may still appear in search results. A sitemap aids discovery without guaranteeing crawling or indexing. Writing “do not train” in llms.txt does not create authentication or access control. Keep private information out of both the file and its linked resources.

What is known, and what remains unproven

The proposal's format and the fact that particular sites publish files can be checked directly. Publication does not prove that a particular AI product always reads the file while answering. Evaluate adoption against that product's own official description of its behavior.

Google's guidance on AI search features says new AI text files are unnecessary for appearing in AI Overviews or AI Mode. That statement should not be generalized into a claim that every AI ignores llms.txt.

Measure your implementation through working links, successful navigation to the right reference, and agreement with the HTML content. You can test your own tool by explicitly providing the URL and checking whether it retrieves the necessary pages. Keep that separate from automatic discovery or citations by external services. For broader decisions about which information to expose, see AI agents and CMS integration.

FAQ

Is llms.txt mandatory?

Implementing the proposal is optional. It can provide an entry point to public resources, but it is not required to publish a normal website or appear in Google's AI search features.

Is llms-full.txt enough on its own?

A full-text file makes it harder to select a small reading scope. A short index and individual Markdown pages let readers choose the full-text version when needed.

Can I generate it without built-in CMS support?

If you can retrieve the public content, your site build can generate the files. Implement permissions, public-version selection, body conversion, and update timing for your own setup.

Will publishing it change AI answers or search rankings?

Publication alone provides no such guarantee. Making a file accessible and verifying whether an external service actually uses or cites it are separate tasks.