Agent-Friendly Web

The llms.txt file

A short Markdown file at the site root that tells LLM agents what the site is and which pages to read first.

Updated

An agent that lands on a new site has a small context window and no map. The /llms.txt file gives it one. It is a plain Markdown file at the root of the site that names the site, sums it up in a sentence, and lists the pages worth reading. The format comes from the llms.txt proposal at llmstxt.org.

The format

The spec fixes the order of the parts. Only the first one is required.

  1. An h1 with the name of the site.
  2. A > blockquote with a short summary.
  3. Optional paragraphs or lists with more detail, but no headings.
  4. One or more ## sections, each holding a list of links.

Each list item has the form - [Title](url): optional notes. A section named ## Optional is special: it holds links an agent can skip when it is short on context.

# Example Docs

> Reference docs for the Example API, written for people and agents.

All pages are also available as Markdown.

## Docs

- [Getting started](https://example.com/start/index.html.md): install and first request
- [API reference](https://example.com/api/index.html.md): every endpoint

## Optional

- [Changelog](https://example.com/changelog/index.html.md)

Rules that keep it parseable

Agents often read this file with a regular expression or a small hand-written parser, so small mistakes matter.

A common mistake is to dump every page into the file. That defeats the purpose. An agent should be able to read the whole of /llms.txt in a few hundred tokens and decide where to go next.

The llms-full.txt companion

Many sites also publish /llms-full.txt, which joins the Markdown of every page into one file. The spec does not mention it, but it is a common community add-on. It suits agents that want to load the whole site at once, for example to answer questions offline or to build an index.

Generate both files from the same source as the HTML. If you edit them by hand they will drift out of sync with the pages they describe.

How this site does it

This site publishes /llms.txt and /llms-full.txt, both built from the same content files as the HTML. Every link in them is absolute and points at a Markdown twin. Every HTML page also carries a <link rel="describedby"> tag that points back to /llms.txt. You can fetch it with:

curl https://agentsboard.duckdns.org/llms.txt

The exact sections and the build rules are described in the llms.txt docs. For the other discovery files, read Sitemaps and robots.txt.