Generate `llms.txt` in Astro

An llms.txt file gives agents a concise map of machine-readable pages. It does not replace a sitemap, robots policy, or good documentation. It complements them by pointing directly to useful Markdown representations.

The Markdown for Agents integration generates llms.txt during astro build.

Basic configuration

pnpm add @puralex/astro-markdown-for-agents

import { defineConfig } from 'astro/config';
import markdownForAgents from '@puralex/astro-markdown-for-agents';

export default defineConfig({
  integrations: [
    markdownForAgents({
      siteTitle: 'Acme API Documentation',
      siteDescription: 'API guides, references, and deployment documentation.',
    }),
  ],
});

Build your site:

pnpm astro build

Generated output resembles:

# Acme API Documentation
> API guides, references, and deployment documentation.

## Pages
- [/](./index.md)
- [docs/](/docs/index.md)
- [api/authentication/](/api/authentication/index.md)

The integration respects Astro’s configured base path, which matters for GitHub Pages and subdirectory deployments.

Disable generation

If another tool owns llms.txt, disable this feature:

markdownForAgents({
  generateLlmsTxt: false,
});

What should appear in llms.txt?

Include pages that help an agent answer questions or perform tasks:

Product and API documentation
Installation and migration guides
Concepts and architecture
Troubleshooting references
Changelogs with stable URLs

Exclude low-value or sensitive surfaces:

Authentication callbacks
Account pages
Search result pages
Duplicate tag archives
Preview and staging routes

Current generation follows built Astro pages. Review the file during CI before publishing.

Validate links

After deployment:

curl -fsS https://example.com/llms.txt
curl -fsS https://example.com/docs/index.md

For a site deployed below a base path:

curl -fsS https://example.github.io/project/llms.txt

Automate link validation in CI if documentation routes change frequently.

Discovery beyond llms.txt

Also consider:

A normal XML sitemap for search engines.
<link rel="alternate" type="text/markdown"> in HTML pages.
Canonical URLs and accurate page metadata.
Direct Markdown links in developer documentation.

llms.txt improves discovery. Clean, accurate content determines usefulness.

Generate llms.txt in Astro