Guides

Short, sourced explainers on how AI answer engines reach and read your site. Each claim links to the documentation it comes from.

• Why ChatGPT can't see your site

  The four usual reasons ChatGPT never mentions or links your site: robots.txt, a firewall or CDN block, content that only appears after JavaScript runs, and broken URLs. How to check each one.

• robots.txt vs Cloudflare AI bot blocking

  robots.txt states a preference; Cloudflare's AI bot settings enforce one at the edge. How the two interact, where they conflict, and how to set them so the crawlers you want can still get in.

• llms.txt: do you need it

  What llms.txt is, who uses it today, and an honest answer on whether it helps you appear in ChatGPT, Perplexity or Google's AI answers. It's optional, and its effect on AI answers isn't proven.