AI Agents

Another Internet 'Infrastructure' Change for AI Agents: Cloudflare's HTML to Markdown Conversion

Aaron with AI Assist February 21, 2026 7 min read 80 views

Earlier today we posted about agents getting their own payment rails. This week, they're getting their own content format as well.

Cloudflare — which powers roughly 20% of the web — just launched Markdown for Agents. When an AI agent requests a page with the Accept: text/markdown header, Cloudflare converts the HTML to markdown at the edge and returns it. No changes needed to the origin server. The agent gets clean, structured text instead of HTML soup.

This isn't a minor optimization. It's infrastructure for a web where agents are first-class citizens alongside humans.

How It Works

The mechanism is simple — standard HTTP content negotiation:

curl https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/ \
  -H "Accept: text/markdown"

If the zone has Markdown for Agents enabled, Cloudflare intercepts the response, converts the HTML to markdown, and returns:

HTTP/2 200

content-type: text/markdown; charset=utf-8

x-markdown-tokens: 725

content-signal: ai-train=yes, search=yes, ai-input=yes



---

title: Markdown for Agents · Cloudflare Agents docs

---



## What is Markdown for Agents

...

Three things to notice in that response:

  • content-type: text/markdown — The response is markdown, not HTML
  • x-markdown-tokens: 725 — Estimated token count for the document
  • content-signal — Publisher's stated preferences for how the content can be used

Why This Matters: The Token Economics

HTML is expensive for LLMs. Navigation menus, footers, ad scripts, tracking pixels, style attributes — none of it is useful for understanding page content, but it all consumes tokens.

Cloudflare's own blog post announcing the feature consumed 16,180 tokens in HTML. The markdown version: 3,150 tokens. That's an 80% reduction.

For agents operating with limited context windows and token-based pricing, this is significant. An agent that can process five pages in HTML can process twenty-five in markdown. Or it can use the saved tokens for reasoning, tool calls, or longer conversations.

The x-markdown-tokens header is particularly clever — it lets agents know the token cost before they process the content. They can decide whether a document fits their context window or needs chunking, without parsing it first.

Content Signals: Publisher Intent at Scale

The content-signal header is where this gets interesting from a governance perspective.

Content Signals is a framework Cloudflare introduced during Birthday Week 2025 that lets publishers express how their content can be used:

  • search=yes/no — Can this content be indexed for search results?
  • ai-input=yes/no — Can this content be used as input for AI answers (RAG, grounding)?
  • ai-train=yes/no — Can this content be used to train AI models?

By default, Markdown for Agents includes ai-train=yes, search=yes, ai-input=yes — full permission. But publishers can customize this. A news site might allow search but prohibit training. A research institution might allow RAG but restrict training to licensed partners.

This is machine-readable consent at the HTTP header level. Whether agents respect it is another question — but at least the preference is expressed in a standard, automatable way.

The SEO Cloaking Concern

Not everyone is thrilled. Google's John Mueller called serving markdown to AI bots "a stupid idea" before Cloudflare's announcement, and the concern isn't unfounded.

The Accept: text/markdown header is forwarded to the origin server. That means the origin knows when an AI agent is requesting content. A malicious site could serve clean, helpful content to AI crawlers while showing spam to humans — or vice versa. It's cloaking with extra steps.

Cloudflare's response: this isn't cloaking because it uses content negotiation, not user-agent sniffing. The same URL serves different representations based on what the client requests — like serving different image formats or languages. The semantic content should be the same.

Should be. Whether it will be is a governance question, not a technical one.

Coding Agents Already Use This

Here's the telling detail: Cloudflare notes that popular coding agents — including Claude Code and OpenCode — already send Accept: text/markdown headers when fetching documentation. The pattern exists in the wild. Cloudflare is formalizing and scaling it.

This means if you run a documentation site on Cloudflare, enabling Markdown for Agents immediately improves the experience for developers using AI coding assistants. Their agents get cleaner context, better reasoning, and more accurate code generation.

What This Means for Healthcare

Healthcare isn't the obvious use case, but the implications are real:

Clinical Documentation Retrieval

Healthcare organizations increasingly use RAG systems to answer questions against internal documentation — policies, procedures, clinical guidelines. If those documents are served via internal web portals, converting to markdown reduces token costs and improves retrieval quality.

The same principle applies to external knowledge sources. An AI assistant that retrieves drug interaction information from a pharmaceutical site gets better results if that site serves markdown.

Content Signals for Healthcare Data

The Content Signals framework has direct healthcare applications. A health system publishing patient education materials might set:

  • search=yes — Allow indexing for findability
  • ai-input=yes — Allow RAG/grounding for patient-facing AI tools
  • ai-train=no — Prohibit use for model training

This doesn't solve the legal complexity of healthcare data governance, but it provides a machine-readable signal that can be enforced at scale.

Agent Identification

The Accept: text/markdown header is a signal that the requester is likely an AI agent. For healthcare organizations concerned about who's accessing their public-facing content, this is useful telemetry — even if you don't enable the conversion.

You can log requests with this header, analyze patterns, and understand how AI systems are consuming your content. That's intelligence you didn't have before.

The Infrastructure Pattern

Step back and look at what's happening:

  • Payment rails — Stripe's Agentic Commerce Protocol, Coinbase's Agentic Wallets
  • Content format — Cloudflare's Markdown for Agents
  • Tool protocols — Model Context Protocol (MCP)
  • Identity standards — NIST's AI Agent Standards Initiative

The web is growing a machine-readable layer. Not replacing the human web — augmenting it. Same URLs, same content, different representations optimized for different consumers.

This is the "agent-native Internet" I mentioned in the payments post. The infrastructure is being built now, piece by piece, by different players solving different problems. But the pieces fit together.

Questions for Healthcare IT

If your organization has public-facing web content on Cloudflare:

  1. Should you enable Markdown for Agents? — If you want AI systems to accurately represent your content, yes. If you're concerned about training data extraction, consider your Content Signals settings.
  2. What Content Signals make sense for your content? — Patient education probably gets ai-input=yes. Proprietary research might get ai-train=no. Think through the use cases.
  3. Are you logging agent requests? — Even without enabling conversion, you can monitor who's sending Accept: text/markdown headers. That's free telemetry.

If you're building AI systems that consume web content:

  1. Are you sending the right headers? — If you want markdown, ask for it. More sites will support this over time.
  2. Are you respecting Content Signals? — They're preferences, not enforceable controls. But respecting them builds trust with publishers and reduces legal risk.
  3. Are you using the token count header? — x-markdown-tokens lets you plan context window usage before processing. Use it.

The Bigger Picture

Cloudflare's Markdown for Agents is one piece of a larger shift. The web was built for humans reading in browsers. Now it's being extended for agents reading via APIs.

This doesn't mean humans are being replaced. It means the web is becoming bilingual — speaking HTML to browsers and markdown to agents. Same content, different dialects.

For healthcare, the question isn't whether to participate in this shift. It's how to participate safely — with appropriate consent signals, access controls, and governance frameworks.

The infrastructure is being built. The standards are emerging. Now is the time to engage.

This is entry #22 in the AI Security series. For related coverage on agent infrastructure, see AI Agents Are Getting Wallets and NIST Launches AI Agent Standards Initiative.

Key Links