The llms.txt Guide: How to Tell AI Search Engines What Your Site Does
Adeyinka Adefila
Founder, Distro · June 4, 2026
An llms.txt file is a plain text file placed at the root of your website that describes your content in a format designed for large language models to read. Think of it as robots.txt for AI. While robots.txt tells search engine crawlers which pages to index, llms.txt tells AI models like ChatGPT, Claude, and Perplexity what your site does, who it is for, and which content to prioritize when generating answers. Most websites don't have one yet. Adding one takes 10 minutes and can significantly improve your chances of being cited in AI search results.
It's one of the highest-leverage things you can do for AI visibility right now, precisely because so few sites have caught on. Ten minutes of work puts you ahead of almost everyone in your category.
Key Takeaways
- llms.txt is a plain text summary of your site, written for AI models
- It lives at your root domain, right next to robots.txt
- Include what you do, who it's for, features, pricing, and key links
- Keep it between 500 and 2,000 words, factual and clean
- It's an emerging convention, not yet an official standard, but adoption is rising
What llms.txt Is and Why It Exists
llms.txt exists because AI models need a clean, authoritative source of truth about your site, and your homepage HTML isn't always it. Pages are full of navigation, scripts, and marketing copy that a model has to wade through. An llms.txt file hands the model a tidy summary so it doesn't have to guess what matters.
The format borrows the idea from robots.txt: a simple file at a known location that machines check. The difference is the audience. robots.txt talks to crawlers about access. llms.txt talks to language models about meaning.
How AI Search Engines Currently Discover Your Content
Today, models find you through crawling, structured data, and existing indexes like Bing's. They piece together what your site does from your headings, your schema markup, and whatever text they can extract. It works, but it's lossy. Important context gets buried and unimportant text gets weighted.
An llms.txt file shortcuts that process. Instead of inferring what your product does from scattered signals, the model can read a direct statement. That clarity makes accurate citation far more likely.
What Goes in an llms.txt File
Cover six things: a short description of your product, who it's for, your core features with one-line explanations, your pricing, your most important content, and your key links. Write it like you'd explain your company to a smart stranger in two minutes. Factual, specific, no fluff.
Avoid marketing language. The model isn't a customer you need to persuade. It's a librarian deciding how to file and quote you. Give it clean facts, not slogans.
Step by Step: Creating Your llms.txt File
Create a new file named llms.txt. Fill in the template below with your details. Save it and place it at your root domain so it's reachable at yoursite.com/llms.txt. That's the whole process. Here's a template you can copy:
# [Your Product Name]
## What is [Product Name]?
[2-3 sentence description]
## Who is it for?
[Target audience]
## Core Features
- [Feature 1]: [One-line description]
- [Feature 2]: [One-line description]
## Pricing
[Plan details]
## Links
- Homepage: [URL]
- Pricing: [URL]
- Blog: [URL]
- Documentation: [URL]
Where to Place It
Put llms.txt at your root domain, in the same place as robots.txt, so it's available at yoursite.com/llms.txt. Crawlers and models check the root by convention. If you bury it in a subfolder, it won't be found.
If you're on a framework like Next.js, you can serve it from your public folder or as a route. The only rule that matters is the final URL: it has to resolve at the root.
llms.txt vs robots.txt vs sitemap.xml
Each file does a different job, and they work together. robots.txt tells crawlers which pages they may access. sitemap.xml lists your pages so crawlers can find them all. llms.txt explains what your site means so models can describe and cite it accurately. Access, discovery, and meaning. You want all three.
None of them replaces the others. A complete setup has robots.txt allowing the right crawlers, a sitemap listing your pages, and an llms.txt summarizing what it all adds up to.
Common Mistakes
The usual errors are easy to avoid. Too short, and the model learns nothing useful. Too long, and the signal drowns. Keyword stuffing, which makes you look spammy and helps nobody. And leaving out contact or product basics, which makes the file feel like an afterthought. Aim for clear, complete, and concise.
One more: don't write it once and forget it. When your product or pricing changes, update llms.txt too, or you'll have a model confidently citing stale facts about you.
How to Verify AI Can Read Your File
After publishing, visit yoursite.com/llms.txt in a browser to confirm it loads as plain text. Then ask ChatGPT, Claude, or Perplexity about your product and see whether the answers improve. For a structured check, scan your site with the AI citation checker, which confirms whether your file is present and readable.
For the full background, the concept is defined under llms.txt, and the wider strategy lives in the generative engine optimization guide.
Frequently Asked Questions
Is llms.txt an official standard?
It is an emerging convention, not an official standard like robots.txt. However, AI companies are increasingly recognizing it, and having one gives AI models a clean, structured source of information about your site.
Will llms.txt help me rank on Google too?
Not directly. Google uses its own crawling and indexing systems. But the act of creating a clear product description and content summary can improve your overall content clarity, which benefits both Google and AI search.
How long should my llms.txt be?
Between 500 and 2,000 words. Long enough to cover what your product does, who it is for, and your key content. Short enough that an AI model can process it in one pass. Avoid repeating the same information or stuffing keywords.
Check if AI search engines can find your site with the free AI citation checker at www.usedistro.com/tools/ai-citation-checker.