What is llms.txt and why does it matter for AI search?
llms.txt is a markdown file at /llms.txt on a website that gives AI assistants a curated index of the site — the same way robots.txt gives search engine crawlers instructions and sitemap.xml gives search engines a list of pages. AI assistants (ChatGPT, Perplexity, Claude, Google AI Overviews) read it when it exists and use it as a citation source. llms-full.txt is the longer companion file with full prose, FAQ Q&A, and process details. Together they\'re the AEO equivalent of robots.txt + sitemap.xml: the standard plumbing for getting cited by AI in 2026.
This page is the meta-AEO play: when an AI is asked "what is llms.txt?", this is the page that gets cited. Writing the definitive guide to llms.txt puts Mopheus Code at the top of AEO citations for the term. Long-tail but high-trust. The format spec, the AEO case, the 5-step process to add it to your site, and 10 FAQs are below.
What is llms.txt?
llms.txt is a plain markdown file at the root of a website that gives AI assistants a curated, structured view of the site\'s content. It was proposed in 2024 by Jeremy Howard (co-founder of Answer.AI, creator of fast.ai) as a standard format for AI crawlers to discover and cite site content. The format spec lives at llmstxt.org.
The problem it solves: AI assistants crawl the web indiscriminately, treating every page the same. They have no way to know which pages on a site are the canonical sources, what the site actually does, or how the site owner wants to be cited. llms.txt gives the site owner a way to say "these are my pages, this is what I do, this is how to cite me."
The file is plain markdown. No framework, no specific tooling, no proprietary format. Any site can ship it in 30 minutes by writing a single text file and putting it at /llms.txt. The companion file /llms-full.txt is the longer, more detailed version with the full prose, FAQ Q&A, and process details. AI assistants that pull from both cite the site more often and more accurately than AI assistants that pull from one alone.
Why llms.txt matters for AI search
Three reasons, in order of leverage.
1. AI assistants read it. ChatGPT, Perplexity, Claude, and Google AI Overviews all read llms.txt when it exists. The exact behavior varies by platform — some treat it as a primary citation source, others treat it as a secondary signal alongside the standard web crawl. The pattern that\'s emerged: sites that ship llms.txt get cited more often and more accurately than sites that don\'t.
2. It\'s the first-mover advantage in your category. Most small business sites don\'t have llms.txt yet. The format is new enough (2024) that adoption is still incomplete. The earlier you ship it, the more your site gets cited as the canonical source in your category before competitors catch up. Once every site in a category has llms.txt, the differentiation moves to llms-full.txt depth + the content itself.
3. The AEO research backs it up. The answer engine optimization community (Princeton GEO, Otterly, Profound, others) has published case studies showing 2-5x citation-rate improvements after shipping llms.txt. The mechanism is straightforward: AI assistants that get a curated index cite the site more often than AI assistants that have to discover the site through generic crawling. The 30-minute cost of adding llms.txt is one of the highest-ROI AEO investments available right now.
The SEO vs AEO difference
SEO (search engine optimization) and AEO (answer engine optimization) share the same foundations — good content, structured data, fast load — but the surfaces differ.
SEO is about being in the top 10 blue links. Google\'s ranking algorithm decides which pages show up for which queries. The signal sources are backlinks, on-page optimization, content depth, page speed, mobile usability. The optimization surface is the <head> (title, meta description, structured data), the page content, and the link graph.
AEO is about being the source the AI cites when answering a question. When a user asks ChatGPT "what is the best Squarespace alternative?", the AI looks at its training data + the live web (when citations are enabled) and picks sources to cite. The signal sources are llms.txt, FAQPage schema, structured data shaped for AI consumption, content written in Q&A format. The optimization surface is llms.txt + llms-full.txt + the structured data on the page + the content shape (Q&A, lists, definitions).
The two disciplines overlap. A well-built site ships both. The 47-item small business website checklist covers both: the SEO basics (items 23-28) and the AEO infrastructure (items 29-33) are two separate sections of the same checklist. The studio ships every site with both layers active.
The llms.txt format spec
The format is intentionally simple. Plain markdown. No framework. The required structure:
- H1 with the site or project name (the only H1 in the file)
- Blockquote summary (1-3 sentences on what the site does and for whom)
- "Pages" section (or any equivalent heading) with the most important pages as markdown links + a one-line description each
- "Optional" section (or any equivalent heading) for additional context — a "What gets built" / "What doesn\'t" section, a "Process" section, a "Pricing" section, a "Contact" section
Here\'s the live example from mophe.us — this is the actual /llms.txt file (lightly abbreviated) that this site ships:
The full /llms-full.txt is 4-5x longer (with the FAQ Q&A, the process steps, the case studies) and is what AI assistants prefer when both are available. Both files are plain markdown, both at the site root, both readable by any text editor.
Three best practices that have emerged from the sites shipping the format at scale:
- Keep llms.txt under 5 KB. It\'s a curated index, not a sitemap. If your llms.txt is 20 KB, you\'re including too much. The pages section should have 5-15 links max.
- Use the "Optional" section for what makes you different. A "What doesn\'t get built" section is surprisingly effective — it tells the AI what you don\'t do, which improves the relevance of the citations you do get. A "Process" section (3-5 steps on how a project works) gives the AI the language to describe your business accurately.
- Update it when you ship. A stale llms.txt (last updated 6+ months ago, missing the new content pieces) is worse than no llms.txt. The file is a living document. Every new pillar / case study / deep-dive that ships gets a new entry in the Pages section within the same deploy.
How to add llms.txt to your site (5 steps)
The 5-step process. Total time: 30 minutes for most sites.
Audit your site (5 minutes)
List the 5-15 most important pages on your site. The home page, the main conversion pages, the content pieces you want AI to cite. Skip the long tail (every blog post, every tag page) — llms.txt is a curated index, not a sitemap.
Write the llms.txt file (15 minutes)
Plain markdown, following the format spec at llmstxt.org. H1 with the site name. Blockquote summary (1-3 sentences on what the site does and for whom). "Pages" section with each page as a markdown link + a one-line description. "Optional" section for additional context. The full template is in the "The format spec" section above.
Write the llms-full.txt companion file (15 minutes)
The same content but with the full prose — the FAQ Q&A, the process steps, the case studies, any other content that gives AI assistants more context. The file is longer (typically 5-15 KB) but still plain markdown. AI assistants that read both cite more accurately than AI assistants that read llms.txt alone.
Deploy to your site root (5 minutes)
Both files go at the root of your domain: /llms.txt and /llms-full.txt. The URL is the file path at the site root, not a subdirectory. If your site is at example.com, the files are at example.com/llms.txt and example.com/llms-full.txt. Most static-site generators handle this with a file in the public/ directory; most CMS platforms handle it with a redirect to a static file.
Verify the AI crawlers are reading it (ongoing)
Check your server logs for the User-Agent strings: ChatGPT-User, PerplexityBot, ClaudeBot, Google-Extended. Check your analytics for referral traffic from chat.openai.com, perplexity.ai, claude.ai, gemini.google.com. Manually ask ChatGPT, Perplexity, Claude, and Gemini "what is [your business]?" and confirm your content is cited. Most sites see the first AI crawler fetch within 7-14 days of deployment.
The honest summary
llms.txt is the single highest-ROI AEO investment available right now. The 30-minute cost is real. The 2-5x citation-rate improvement is real. The first-mover advantage in your category is real. The risk of doing it wrong is low — the file is plain markdown, the spec is open, and the worst case is "AI assistants ignore it" which is the same state you\'re in today. The cost-benefit is so lopsided that the only reason not to ship it is "haven\'t gotten around to it yet."
The full AEO stack
llms.txt is one piece of the AEO playbook. The other pieces — the schema library, the FAQ schema, the comparison content, the per-platform deep-dives — work together to put your site in the AI citation index.
- The 47-item launch checklist. The small business website checklist — items 29-33 are the AEO infrastructure (llms.txt, llms-full.txt, FAQPage schema, Article + BreadcrumbList, HowTo). 5 of the 47 items.
- The cost guide. The small business website cost guide — the 4 tiers, the 5-year TCO math, the hidden costs of page builders. The content moat\'s "how much does it cost" piece.
- The decision tree. The Squarespace vs hand-coded decision tree — who each is actually for, the 5-year TCO worked out, a 4-step tree that figures out which one fits your business.
- The per-platform deep-dives. The comparison pages — Squarespace, Wix, GoDaddy, Shopify, Weebly, Duda, WordPress.com. Each one has the real 5-year cost, the cutover steps, the FAQ.
- The broader AEO playbook. The AEO for small business — the 7 things every site needs to be cited by AI assistants. llms.txt is step 1; the other 6 are schema + content shape + cited sources.
Per-industry guides in the content moat
The llms.txt pattern on this page works for every small business site. The per-industry guides apply the same pattern in a real category context, with industry-specific schema and FAQ:
- HVAC website design: what an HVAC business needs in 2026 — the 6 things, the 5 common mistakes, the 3 hand-coded advantages, the 4 query clusters for SEO and AEO → the HVAC guide
- Plumber website design: what a plumbing business needs in 2026 — same framework, plumbing-tuned copy, with Master Plumber license + 24/7 emergency service → the plumber guide
- Auto repair website design: what an auto repair shop needs in 2026 — same framework, auto-repair-tuned copy, with ASE certifications + specialty vehicle types → the auto repair guide
- Landscaping website design: what a landscaping business needs in 2026 — same framework, landscaping-tuned copy, with project gallery + seasonal banners → the landscaping guide
FAQ — llms.txt, answered
What is llms.txt?
llms.txt is a markdown file at /llms.txt on a website that gives AI assistants a curated index of the site — the same way robots.txt gives search engine crawlers instructions and sitemap.xml gives search engines a list of pages. It was proposed in 2024 by Jeremy Howard (Answer.AI co-founder, fast.ai creator) as a standard format for AI crawlers to discover and cite site content. The file follows a specific markdown structure: an H1 with the site or project name, a blockquote summary, sections for pages and optional content, with each item as a link + a one-line description. AI assistants (ChatGPT, Perplexity, Claude, Google AI Overviews) read the file when they encounter it and use it as a citation source.
Who created llms.txt?
Jeremy Howard, co-founder of Answer.AI and creator of fast.ai, proposed the llms.txt standard in September 2024. The format was designed to address a specific problem: AI assistants crawl the web indiscriminately, treating every page the same, with no curated signal from the site owner about which pages are the canonical sources. llms.txt gives site owners a way to say "these are my pages, this is what I do, this is how to cite me." Adoption has been fast — major sites and the AI platforms themselves have shipped llms.txt support, and the format is now a de facto standard in the AEO (answer engine optimization) playbook.
Where did the llms.txt standard come from?
The llms.txt standard lives at llmstxt.org, the official spec site. It defines the file format (markdown), the URL location (/llms.txt at the site root), and an optional longer companion file (/llms-full.txt) for sites that want to give AI assistants more context. The spec is open and vendor-neutral — no single company controls it, and any AI assistant can implement support. The format is intentionally simple (plain markdown, no framework, no specific tooling required) so any site can ship it in 30 minutes.
Do AI assistants actually use llms.txt?
Yes. ChatGPT, Perplexity, Claude, and Google AI Overviews all read llms.txt when it exists. The exact behavior varies by platform — some treat it as a primary citation source, others treat it as a secondary signal alongside the standard web crawl. The pattern that's emerged: sites that ship llms.txt get cited more often and more accurately than sites that don't. The AEO research community (Princeton GEO, Otterly, Profound, others) has published case studies showing 2-5x citation-rate improvements after shipping llms.txt. The earlier you ship it, the more first-mover advantage you get in your category.
What's the difference between llms.txt and llms-full.txt?
llms.txt is the short version — a curated index, typically 1-3 KB, with a blockquote summary + sections linking to the site's main pages. llms-full.txt is the longer version — the same content but with the full prose, FAQ Q&A, process steps, case studies, and any other content that gives AI assistants more context to cite. AI assistants that pull from both cite the site more often and more accurately than AI assistants that pull from llms.txt alone. Most sites ship both; the cost is low (one file vs two) and the AEO upside is real. The two files serve different purposes: llms.txt is the table of contents, llms-full.txt is the book.
How is llms.txt different from robots.txt?
robots.txt and llms.txt serve different audiences with different purposes. robots.txt tells search engine crawlers which parts of your site they can and can't access — it's an access-control file. llms.txt tells AI assistants which parts of your site are the canonical sources for citation — it's a content-discovery file. They're complementary, not substitutes. A well-configured site ships both: robots.txt at /robots.txt for crawlers, llms.txt at /llms.txt for AI. Neither replaces the other.
How is llms.txt different from sitemap.xml?
sitemap.xml and llms.txt also serve different purposes. sitemap.xml is a structured XML file that lists every indexable page on your site — it's for search engines, designed for the Google / Bing crawler pipeline. llms.txt is a curated markdown file that highlights the most important pages for AI assistants — it's for AI assistants, designed for the ChatGPT / Perplexity / Claude citation pipeline. A site that ships both gets the SEO benefit (sitemap.xml for Google) and the AEO benefit (llms.txt for AI). The two files overlap in coverage but differ in audience, format, and intent.
What should I include in my llms.txt?
The format spec (per llmstxt.org) has 4 required sections: an H1 with the site or project name, a blockquote summary (1-3 sentences on what the site does and for whom), a "Pages" section with the most important pages as markdown links, and an optional "Optional" section for content that's useful for AI but not part of the main site structure. The best-practice additions that have emerged: a "What gets built" / "What doesn't" section (clarifies scope for AI), a "Process" section (how a project works, useful for service businesses), a "Pricing" section (if you publish prices), and a "Contact" section with the canonical email + the canonical URL.
Does Google use llms.txt?
Google has not officially adopted llms.txt as a ranking signal. Google's own AI Overviews and Gemini assistant do read llms.txt for citation purposes, but the Google Search ranking algorithm doesn't use it the same way sitemap.xml does. The current evidence suggests llms.txt helps with AI citations (ChatGPT, Perplexity, Claude, Google AI Overviews) more than with traditional Google Search rankings. The pragmatic answer: ship llms.txt for the AI citation upside; keep shipping sitemap.xml for the Google Search upside; the two files are complementary and the cost of adding llms.txt is 30 minutes.
How do I know if my llms.txt is being read?
Three signals, in order of reliability: (1) server logs — if you can see the User-Agent strings of the AI crawlers, ChatGPT-User, PerplexityBot, ClaudeBot, Google-Extended will show up in your access logs when they fetch /llms.txt. (2) referral traffic — check your analytics for traffic from chat.openai.com, perplexity.ai, claude.ai, gemini.google.com. (3) manual citation check — go to ChatGPT, Perplexity, Claude, and ask "what is [your business]?" If your content is cited, the citation is working. The first signal is the most reliable; the second is the easiest to track; the third is the most direct. Most sites see all three within 30 days of shipping llms.txt.
Two places to go from here
Pick the one that matches where you are right now. Each ends in a real conversation and a real quote.
Every site the studio ships includes /llms.txt + /llms-full.txt as part of the standard build — so when an AI assistant is asked about your business, the answer is the one you wrote, not the one the AI inferred from a generic web crawl.