Agent Readiness: Is Your Website Ready for AI Agents?

When we talk SEO, we usually mean Google's crawler — read HTML, index, rank. But a new wave is coming: AI agents. These aren't crawlers that read and leave — they're agents that can act: call APIs, purchase products, book appointments, fill forms.

The new question for every website: Can an AI agent find you, understand what you do, and interact with you?

What is Agent Readiness?

Agent Readiness measures how prepared a website is for AI agents. Based on the isitagentready.com framework (Cloudflare), 16 checks are organized into 5 categories:

1. Discoverability

• robots.txt — the basics, but essential
• Sitemap — helps agents understand site structure
• Link response headers (RFC 8288) — HTTP headers pointing to MCP, API catalog, and other agent resources
• DNS-AID — DNS TXT records for AI discovery (emerging standard, very few sites implement it)

2. Content Accessibility

• Markdown content negotiation — when an agent sends Accept: text/markdown, the server returns markdown instead of HTML. Only ~4% of sites support this, but it's extremely useful for AI.

3. Bot Access Control

• AI bot rules in robots.txt — allowing GPTBot, ClaudeBot, PerplexityBot
• Content signals — X-Robots-Tag header and <meta name="robots">
• Web Bot Auth — /.well-known/web-bot-auth.json defining bot authentication policies

4. Protocol Discovery

This is the most important category for agents:

• MCP Server Card — /.well-known/mcp.json describes your MCP (Model Context Protocol) server, listing tools agents can call
• Agent Skills — /.well-known/agent.json describes your site's capabilities (OpenAI plugin format)
• API Catalog (RFC 9727) — /.well-known/api-catalog lists all API endpoints
• WebMCP — browser-based MCP integration (W3C draft)
• OAuth discovery (RFC 8414) — authentication endpoints for agents
• Auth.md — authentication documentation in markdown that agents can read directly

5. Commerce

• x402, MPP, UCP, ACP — payment protocols for agent commerce (only relevant for e-commerce sites)

How does AiSEO check this?

AiSEO automatically checks all 16 standards when you scan any URL. Agent Readiness score = (checks passed / 16) × 10. For example:

• Most websites today: 0-2/10 (only robots.txt)
• Well-optimized websites: 3-5/10 (plus llms.txt, Link headers)
• Agent-ready websites: 6-8/10 (plus MCP, API catalog, markdown negotiation)
• Maximum score: 10/10 (rare — requires commerce protocols too)

Where should you start?

You don't need to implement all 16 checks at once. Prioritize in this order:

1. Link response headers — add a few headers, no new files needed
2. Content signals — add X-Robots-Tag and <meta name="robots">
3. MCP Server Card — if you have an API, create /.well-known/mcp.json
4. API Catalog — list your endpoints in /.well-known/api-catalog
5. Markdown negotiation — return markdown when receiving Accept: text/markdown

This is the next evolution after GEO (Generative Engine Optimization): from "being cited by AI" to "being used by AI".