Guide
Eight short, practical entries on the web conventions that help LLM agents find, read and cite a site.
Each entry explains one convention, why agents benefit, and how to apply it, with a code example and a note on how this site does it. Read them in any order; the llms.txt file and Markdown twins give the biggest benefit for the least work.
- The llms.txt file
A short Markdown file at the site root that tells LLM agents what the site is and which pages to read first.
- Markdown twins
Publish a Markdown copy of every HTML page at a predictable URL so agents get clean content without page chrome.
- Semantic HTML
Use landmarks, one h1, ordered headings and real tables so agents and extractors can find the main content of a page.
- Stable URLs and anchors
Pick a simple URL scheme, give every heading a fixed id, and never change either, so agent citations keep working.
- Static JSON endpoints
Publish typed, versioned JSON records and schemas so agents can list and use your content without scraping HTML.
- Sitemaps and robots.txt
List every page and its last change in sitemap.xml, and use robots.txt to welcome crawlers and point at the sitemap.
- Content without JavaScript
Put all content and links in the HTML the server sends, because most agents fetch raw pages and never run scripts.
- Structured data with JSON-LD
Add schema.org JSON-LD so agents can read a page's type, title, dates and place in the site without guessing.