The /.well-known directory as an AI discovery surface

Past discovery was pages and links. The current layer is HTML metadata plus JSON-LD. The emerging layer is /.well-known/* — fixed paths an agent fetches directly without parsing your site. RFC 8615 reserved that path prefix in May 2019, and the IANA Well-Known URI Registry at iana.org/assignments/well-known-uris holds the canonical list of registered suffixes. AI clients now read several of those entries to learn how to authenticate against your APIs and who to contact about vulnerabilities.

Why agents care about /.well-known

Crawling HTML is expensive and brittle. A fixed path with a JSON payload is the opposite. One HTTP GET, parse, done. That is why every OAuth client already hits /.well-known/openid-configuration instead of hunting through documentation, and why RFC 9728 (OAuth 2.0 Protected Resource Metadata, April 2025) put authorization discovery on the same footing for resource servers. The Model Context Protocol's authorization spec from late 2025 made RFC 9728 mandatory for remote MCP servers. That single decision moved /.well-known/oauth-protected-resource from an obscure metadata endpoint to something every AI client is now expected to fetch on a 401 response.

The pattern generalizes. If you want agents to find a stable signal about your site, a registered well-known URI beats an HTML element a crawler has to scrape and interpret.

The registry, in plain terms

RFC 8615 defines two rules worth knowing. The path lives at /.well-known/<suffix> on HTTPS, HTTP, WS, and WSS origins. The suffix has to be registered with IANA as either permanent or provisional. Section 1.1 also forbids putting the actual user-facing resource at the well-known URL. The endpoint is metadata or a redirect, not the destination. The change-password spec follows that rule precisely: GET /.well-known/change-password returns a 302, 303, or 307 to your real password-change page.

The registry has grown past 100 entries. Most AI tooling cares about fewer than ten of them.

The paths an AI client will actually fetch

A few specifics worth pinning down. security.txt must be served over HTTPS as text/plain and must include a Contact: field plus an Expires: timestamp. The OIDC discovery document has to be returned as JSON over HTTPS, and the issuer value inside the document has to match the URL the client used to fetch it — otherwise clients reject it as an impersonation attempt. RFC 9728 documents work the same way: JSON over HTTPS, optional JWT signing as application/oauth-protected-resource-jwt. CORS matters too. Browser clients fetching these endpoints will fail without Access-Control-Allow-Origin set, even when curl works fine.

The MCP and AI-agent angle

The Model Context Protocol spec's late-2025 authorization revision makes the discovery dance explicit. An MCP client sends an unauthenticated request. The server replies with HTTP 401 and a WWW-Authenticate header pointing at /.well-known/oauth-protected-resource. The client fetches that JSON, reads the authorization_servers array, fetches each one's /.well-known/oauth-authorization-server document per RFC 8414, and runs the OAuth 2.1 flow. No human reads any docs in that loop.

The MCP card story is messier. Practitioners have proposed /.well-known/mcp.json alongside /.well-known/mcp/server.json and related shapes. None is registered with IANA. Until one is, the safer move is what the MCP discovery piece recommends: list your MCP server URL inside /llms.txt under a ## MCP section and register with the MCP Registry. Publishing a well-known MCP card in addition costs nothing if your client target audience expects it.

ai-plugin.json is a useful cautionary tale. OpenAI deprecated the ChatGPT plugin system on April 9, 2024, and the manifest at /.well-known/ai-plugin.json no longer drives any production traffic from ChatGPT. The pattern lived on with the same path prefix and JSON manifest shape, and it influenced what MCP discovery looks like today. Sites with old ai-plugin.json files have nothing to gain from keeping them; delete or ignore. Nothing reads them in 2026.

What every site should ship

Three positions. First, every public site should publish /.well-known/security.txt per RFC 9116. The cost is one text file with a Contact: line and an Expires: field at least one year out. The benefit is a working channel for security researchers and the AI audit tools that now look for it. Sites without one fail the audit and deserve to.

Second, any site with OAuth or OIDC sign-in should already publish /.well-known/openid-configuration. Most managed identity providers serve it for you out of the box. If you operate APIs that expect OAuth-authenticated agent traffic, add /.well-known/oauth-protected-resource per RFC 9728. That single document tells MCP clients and OAuth clients which authorization servers to trust without you writing a paragraph of documentation.

Third, tool sites and developer-platform sites should track the MCP discovery convention as it stabilizes. List your MCP endpoint in /llms.txt today. Add a well-known MCP descriptor when the path lands in IANA. Don't invent your own suffix in the meantime — clients won't look for it.

Password managers will also thank you for /.well-known/change-password returning a 302 to your real change-password page. It's a five-minute redirect rule, and the major browser-bundled password managers all honor it.

Verify before shipping

The HTTP details kill these endpoints more often than the JSON content. Wrong Content-Type, missing CORS, redirect to login, expired security.txt, mismatched issuer in the OIDC document — all silent failures from a browser, all hard failures for an agent. Run the audit at isitready.dev against your canonical origin. Each /.well-known/* finding includes the raw response so you can see exactly what an agent sees.