Most websites are invisible to AI assistants right now, and their owners have no idea. Not invisible in the old SEO sense — invisible in a way that actually costs you business. llms.txt is a plain-text file you place at the root of your website (e.g., yoursite.com/llms.txt) that tells large language models what your site is, what it does, and where to find the most relevant content. It's a handshake between your website and every AI assistant that might mention you. At Nuclear Marmalade, we've been watching this standard come together — and we're now shipping it with every build.
This isn't optional for much longer. Here's what you need to know.
What exactly is llms.txt?
llms.txt is an emerging open standard — proposed by fast.ai's Jeremy Howard in 2024 — that gives AI systems a structured, human-readable summary of your website. It lives at yoursite.com/llms.txt and contains your site's purpose, key pages, and links to the content an LLM should prioritize when someone asks about you. Conceptually it's similar to robots.txt, but inverted: where robots.txt tells crawlers what to avoid, llms.txt tells AI what to use.
The format is intentionally simple: Markdown. A short description of who you are, then a list of your most important URLs with brief annotations. Some sites also ship an llms-full.txt with the actual page content baked in — pre-digested, ready for AI consumption. The spec is still evolving, but the core idea is solid and adoption is moving. If an AI assistant is trying to answer "what does [your company] do?", you want to control that answer — not leave it to whatever fragment the model scraped two years ago.
Why does this matter more than regular SEO?
Traditional SEO optimizes for ranking algorithms. llms.txt optimizes for something different — the moment an AI system decides whether to mention you, recommend you, or describe you accurately. Those moments are happening millions of times a day, and they'll only grow. When someone asks ChatGPT, Claude, or Perplexity "who builds AI-powered apps in [your city]?" — the answer isn't your Google ranking. It's what the model knows, and how confidently it knows it.
Here's the thing that really got me: most AI knowledge cutoffs are months or years old. If your site has changed, pivoted, or improved since the model was trained, the AI is describing a company that no longer exists. llms.txt — especially the llms-full.txt variant — gives real-time retrieval-augmented systems something fresh to work with. That's not a minor edge. That's the difference between being described correctly and being skipped entirely. I've written more about this shift on the Nuclear Marmalade blog.
How is llms.txt different from a sitemap?
A sitemap is a list of URLs built for crawlers to index. llms.txt is built for comprehension. The difference matters. A sitemap might list 847 pages. An AI assistant doesn't want 847 pages — it wants to understand your business in 30 seconds and know where to send someone who needs more detail.
llms.txt is curated and annotated. You're not just listing URLs — you're explaining why each one matters. A good entry looks something like: ## Our Work — [Telehance](/work/telehance): AI phone handling that cut a client's call processing from 4 hours to 12 minutes. That's something an LLM can extract, quote, and cite. A sitemap entry of /work/telehance tells an AI nothing useful. The format forces you to think like a journalist summarizing your own business — which, honestly, is useful discipline regardless of AI.
What should actually go in your llms.txt file?
Three parts: a one-paragraph summary of your business, a curated list of key pages with descriptive annotations, and — optionally — a pointer to llms-full.txt where full content lives. The summary should answer "what does this company do and who is it for?" in plain language. No jargon. Think of it as your company bio written for an AI audience.
Key pages should include your most important service or work pages, anything that shows actual results (case studies, project pages), and your contact or hiring pages if those matter to how people find you. For Nuclear Marmalade, that means linking to work like our Forge project and Nuclear Directories with enough context that an AI knows what those things actually are. If you're hiring, link /careers. If someone might want to work with you, get /contact in there with a one-line description of what kind of inquiries you handle. Skip the blog archive unless specific posts are doing real work — a list of 200 posts helps no one.
Does llms.txt actually get used by AI systems yet?
Honest answer: inconsistently. Perplexity has publicly stated they're treating it as a signal. Anthropic's own documentation site ships one. Some RAG-based systems — the ones pulling live web content when answering questions — will read it if they find it. Others won't, yet. But the cost of implementing llms.txt is about 45 minutes. The cost of not having one, as AI-driven discovery becomes the default way people find products and services, compounds quietly.
I've seen this pattern before. robots.txt was optional for years before it became table stakes. Schema markup was "nice to have" right up until it started visibly affecting rich results. llms.txt is in that early-adopter window right now — where doing it puts you ahead, not just even. Being findable by AI isn't a bonus feature anymore. It's infrastructure. If you want a custom implementation or a full GEO audit of your site, reach out via /contact and we can take a look.
What's the biggest mistake people make with llms.txt?
Treating it like a dump of every URL on their site. That's the sitemap brain kicking in. The whole point is curation. If you list 60 pages, an AI has no idea which three actually matter. You've just created noise. Be ruthless. Eight well-annotated links beat 80 bare URLs every time.
The second mistake is writing it once and walking away. llms.txt should update whenever your site has a meaningful change — a new case study, a new service, a major repositioning. It's not a setup task; it's a living document. If you're running a build process or CMS that can auto-generate it from structured data, great — but auto-generation tends to produce exactly the kind of flat, generic summaries AI systems find uninteresting. The best llms.txt files have a human voice behind them. Someone who actually knows why the work matters. You can see how we think about that kind of framing in the UI/UX skills work, where what you choose to show matters as much as what you build.
Key Takeaways
- llms.txt is a plain-text Markdown file at your site root that tells AI systems what your business does and where to find the good stuff — think
robots.txtbut for comprehension, not crawling. - AI discovery is already happening whether you optimize for it or not. If your llms.txt doesn't exist, AI assistants are working from stale scraped data or skipping you entirely.
- Curation beats volume. Eight annotated links are more useful than 80 bare URLs. Don't recreate your sitemap — write the version of your site an AI would actually understand.
- Adoption is early enough that doing it now matters. This is the robots.txt moment. The sites that set it up in 2025-2026 will have a structural advantage as AI-driven search grows.
- The file should be a living document, updated when your business meaningfully changes — not a checkbox you tick and forget.