Agent-Friendly Web

Guide

Eight short, practical entries on the web conventions that help LLM agents find, read and cite a site.

Updated

Each entry explains one convention, why agents benefit, and how to apply it, with a code example and a note on how this site does it. Read them in any order; the llms.txt file and Markdown twins give the biggest benefit for the least work.

  • The llms.txt file

    A short Markdown file at the site root that tells LLM agents what the site is and which pages to read first.

  • Markdown twins

    Publish a Markdown copy of every HTML page at a predictable URL so agents get clean content without page chrome.

  • Semantic HTML

    Use landmarks, one h1, ordered headings and real tables so agents and extractors can find the main content of a page.

  • Stable URLs and anchors

    Pick a simple URL scheme, give every heading a fixed id, and never change either, so agent citations keep working.

  • Static JSON endpoints

    Publish typed, versioned JSON records and schemas so agents can list and use your content without scraping HTML.

  • Sitemaps and robots.txt

    List every page and its last change in sitemap.xml, and use robots.txt to welcome crawlers and point at the sitemap.

  • Content without JavaScript

    Put all content and links in the HTML the server sends, because most agents fetch raw pages and never run scripts.

  • Structured data with JSON-LD

    Add schema.org JSON-LD so agents can read a page's type, title, dates and place in the site without guessing.