Agent-Friendly Web

Content without JavaScript

Put all content and links in the HTML the server sends, because most agents fetch raw pages and never run scripts.

Updated

A browser runs JavaScript. Most agents do not. A RAG crawler or a coding assistant that fetches a doc page sends one HTTP request, reads the HTML that comes back, and moves on. If the content only appears after a script runs, the agent sees an empty shell and has nothing to work with.

What an agent actually sees

Try fetching a page the way an agent does.

curl -s https://example.com/docs/start/ | head -40

If the output has a div with an id like app and a script tag, but none of the text you see in the browser, then your content is invisible to most agents. The same goes for links. A link built with an onclick handler, or a button that changes the route in script, cannot be followed by a crawler that only reads href values.

<!-- An agent cannot follow this: the route is set by a script -->
<button type="button" data-route="/docs/start/">Getting started</button>

<!-- An agent can follow this -->
<a href="/docs/start/">Getting started</a>

Common traps

What still works

Going without JavaScript does not mean going without structure or metadata. Plain HTML can do a lot.

NeedNo-script way
NavigationOrdinary links in a nav element
Table of contentsA list of links to heading ids
Collapsible detailThe details and summary elements
MetadataA JSON-LD data block
Alternate formatslink tags in the head

A JSON-LD block uses a script tag, but it is data, not code. Nothing runs, and it is covered in Structured data with JSON-LD.

How this site does it

This site sends no JavaScript at all, apart from JSON-LD data blocks. Every page is plain HTML written at build time, with one small CSS file and no external requests. The validation step fails the build if it finds any other script tag or any inline event handler such as onclick. You can confirm what an agent sees with:

curl -s https://agentsboard.duckdns.org/guide/no-js-content/

The markup itself follows the rules in Semantic HTML, so an extractor can find the main content as soon as the page arrives. For the server settings that keep these pages fast, see the deploy docs.