llms.txt: do you need it

What llms.txt is, who uses it today, and an honest answer on whether it helps you appear in ChatGPT, Perplexity or Google's AI answers. It's optional, and its effect on AI answers isn't proven.

Every few weeks someone says llms.txt is the new robots.txt and that you need one to show up in AI answers. Here's what it is and what it does, and what the evidence does and doesn't show, so you can decide for yourself.

What it is

llms.txt is a proposal, first published by Jeremy Howard in September 2024 and revised as v2 in August 2026 (llmstxt.org). The idea is simple: put a Markdown file at /llms.txt that gives AI agents a short summary of your site and a curated list of links to clean, readable versions of your important pages. Web pages carry navigation, ads and scripts. The file is meant to give an agent the useful parts without the clutter.

The format is light. The only required part is an H1 with your site or project name. After that comes an optional one-line summary in a blockquote, any notes you like, and sections headed with H2 that list links:

# Acme Widgets

> Acme makes industrial widgets and ships across the US.

## Products
- [Widget A](https://acme.example/products/widget-a): specs and pricing
- [Shipping and returns](https://acme.example/shipping): policies

## Optional
- [Company history](https://acme.example/about)

The proposal also suggests offering Markdown versions of pages at the same URL with .md added, and linking them with standard link relations.

Who uses it

According to llmstxt.org, thousands of sites publish one, several documentation platforms and CMS plugins generate it automatically, and AI labs including OpenAI, Anthropic and Google (for Gemini) publish llms.txt files for their own developer documentation. The proposal's author says it's used most heavily for software docs, where coding agents follow it to find API references. It also notes that Chrome's Lighthouse now checks for one as part of its agentic browsing audits.

What it is not

It isn't a standard that answer engines have committed to, and it isn't an access control. llmstxt.org describes it as a proposal that's open for community input. It's read on demand by agents that choose to look for it, and nothing tells a crawler to fetch it. It also doesn't replace robots.txt: robots.txt says what automated tools may access, while llms.txt offers context and links.

The honest bottom line: there's no public evidence that having an llms.txt file makes ChatGPT, Perplexity, Gemini or Google's AI features mention or cite your site more. OpenAI's crawler documentation explains how to control its crawlers with robots.txt. It doesn't say llms.txt affects whether or how ChatGPT shows your site (OpenAI: Overview of OpenAI crawlers). Anyone who promises rankings from an llms.txt file is guessing.

So, do you need one?

You don't need one. Whether it's worth adding depends on who reads your site.

Worth doing if you have documentation, an API or a product that people ask coding assistants and agents about. That's where llms.txt is actually used today, and it can save an agent from guessing.
Worth doing if it takes you 15 minutes. It's a small text file with no downside beyond keeping its links current.
Not a priority if AI crawlers can't reach or read your site in the first place. A blocked OAI-SearchBot, a firewall returning 403, or a homepage that's empty without JavaScript matter far more. Fix those first.

We follow that order in our own scoring. In AEO Inspector's 100-point rubric, llms.txt is worth 2 points. AI crawler access in robots.txt is worth 20, firewall access 12, and text visible without JavaScript 12. Those weights are our own judgment about what matters most, not a published standard.

If you add one

Keep it short and current. Link the pages that answer the questions people actually ask: what you do, pricing, docs, policies, contact. Write one plain sentence per link. Then test it the way llmstxt.org suggests: give an agent only your llms.txt and see whether it can answer basic questions about your business.

Check your site

The free check below shows whether you have an llms.txt and, if not, drafts one from your own pages that you can paste in. It also checks the robots.txt, firewall and JavaScript issues that matter more.

1 free check a day per person (per browser) · works without JavaScript · no signup