If you’ve spent time wrestling with SEO tools lately, you might’ve come across this mysterious new thing: llms.txt. It sounds techy (because it is), and it’s being hyped as a cheat sheet for AI bots like ChatGPT, Claude or Gemini. But let’s get one thing out of the way up top:
No AI actually uses llms.txt yet. Not ChatGPT. Not Google Bard. Not Perplexity. Not even Bing.
So should you still bother adding it to your site? Or is it just another shiny object in the SEO echo chamber?
Let’s break it down in plain English.
What is llms.txt, really?
Imagine if you could hand an AI assistant a map of your site’s best pages — no fluff, no pop-ups, no JavaScript chaos. Just: “Here’s what matters, go read this.”
That’s the idea behind llms.txt.
- It’s a plain-text file, named exactly that —
llms.txt— and placed in the root of your website. - The format is Markdown (like a README file).
- It lists key URLs on your site, with short descriptions or titles.
- The goal? Help large language models (LLMs) like GPT or Claude get to the good stuff faster.
Think of it like a shortcut. Where a sitemap is made for search engines like Google, llms.txt is made for AI models. Or at least, that’s the theory.
How does llms.txt work with AI tools like ChatGPT or Gemini?
Short answer: it doesn’t. Not yet.
This is where reality hits harder than an update from Google’s John Mueller.
- ChatGPT, Claude, and Bard do not crawl or use
llms.txt. There’s zero evidence in server logs, and AI companies have said as much. - Google explicitly ignores it. Gary Illyes has confirmed that Google won’t crawl
llms.txt, nor use it in AI Overviews or for ranking.
So even if you carefully curate your best articles in the file, it’s like taping them to a lamppost in the middle of the desert — no one’s reading it.
“No AI system currently uses
llms.txt.” – John Mueller, Google
Why was llms.txt created in the first place?
This isn’t a total pie-in-the-sky idea. The standard was introduced in late 2024 by Jeremy Howard (Fast.ai), a respected figure in the machine learning space.
The intention was smart:
- Give AI models cleaner access to important content.
- Avoid them crawling every noisy HTML menu, pop-up and pixel.
- Help site owners highlight their authority pages.
Some platforms like Perplexity.ai, Cloudflare, and Yoast now support it — with Yoast even offering one-click generation of the file.
But here’s the kicker: support ≠ usage.
Just because a platform lets you create the file doesn’t mean any bots are reading it.
Does llms.txt help with SEO?
Nope. Not even a little bit.
- Google’s AI Overviews don’t look at
llms.txt. They only use content that’s already crawled and indexed the traditional way. - It doesn’t boost keyword rankings. Gary Illyes said, in no uncertain terms, that Google treats it like it doesn’t exist.
- It doesn’t signal anything special about your content. There’s no badge or advantage for having it.
In fact, Google’s advice is refreshingly blunt: stick to regular SEO.
So if you’re spending time fine-tuning your llms.txt hoping it’ll bump you into AI snippets — sorry, it won’t.
Should you enable llms.txt anyway?
Now here’s where things get interesting. Because while it’s useless now, there’s an argument for “future-proofing.”
Let’s weigh it up.
✅ Reasons to Enable It
- Easy and low-risk. If you use Yoast SEO, enabling
llms.txtis literally a toggle switch. - Free maintenance. Yoast auto-updates the file weekly with no extra effort.
- Possible AI adoption in the future. If models start supporting it (like Perplexity or ChatGPT plugins), having it ready could save you time later.
- Better LLM comprehension (theoretically). For big sites with lots of documentation or product pages, a curated file could help AI find what matters — if they ever start looking.
❌ Reasons It Probably Doesn’t Matter
- No current AI tools use it. You’re optimising for a hypothetical future.
- No SEO benefit whatsoever. Google doesn’t touch it.
- Could distract from real work. Time spent fiddling with
llms.txtis time not spent writing quality content or fixing site structure. - Duplicate content risks. Some SEOs warn that repeating page summaries might backfire if not handled properly.
Who’s actually using llms.txt right now?
You might hear that “Stripe has one” or “Cloudflare uses it.” That’s true — some tech-forward companies have adopted it.
But usage doesn’t equal effectiveness. It’s like wearing a Bluetooth headset before Bluetooth was widely supported. You look smart. But the thing’s not connected to anything.
A recent analysis from peec.ai found that not a single LLM-driven search engine has confirmed support for llms.txt. That’s a damning insight.
Even Yoast, the loudest advocate, admits this is speculative groundwork — a nice-to-have, not a must-do.
Can llms.txt help differentiate human vs. AI content?
Nope. The file doesn’t care who wrote your content — it just points to pages. Whether you bled over every word or co-wrote it with an AI, llms.txt won’t judge.
So if you’re looking to signal “Hey, this is human-made!”, this file won’t help.
That said, curating what pages go in the file can reflect strategic thinking. If someone (or something) ever does read it, you’re telling them: “These are the pages that matter.”
But again: no one is reading it. Yet.
Is there a best practice if you do enable it?
Yes. Keep it simple and focused. Here’s what to include:
- Your 10–30 most important pages (e.g., top guides, category hubs, service pages).
- Brief one-liner summaries or titles.
- Don’t try to list your whole site — that’s what sitemaps are for.
- If you’re using Yoast, let it auto-generate weekly — set and forget.
Think of it like leaving a note on your fridge in case the robots come by. Is it useful today? No. But it won’t hurt.
FAQ
Q: Will llms.txt improve my Google rankings?
No. Google has confirmed it does not crawl or consider llms.txt for SEO or AI Overviews.
Q: Can llms.txt help ChatGPT answer questions about my site better?
Not at the moment. ChatGPT doesn’t read or crawl llms.txt.
Q: Should I bother with it if I run a small blog?
Probably not. It’s more relevant (theoretically) for large content-heavy sites.
So… do you need it?
Here’s the straight-up takeaway: No, you don’t need llms.txt today.
It’s an emerging standard with zero adoption by the AI tools that matter. But it also won’t hurt to set it up — especially if it’s automated.
Treat it like a digital insurance policy. Don’t expect a payout, but having it in place might help down the track. Just don’t confuse it with real SEO work.
And if you’re chasing results in Google AI Overviews? Stick to the fundamentals: strong internal linking, clean structure, top-shelf content, and a sitemap that sings.
For those curious how internal links and site structure actually influence Google’s AI snippets, this breakdown explains it well.
Meanwhile, if you’re mapping out future AI opportunities for your brand, consider how you surface your most valuable content in structured ways — because that still counts, llms.txt or not.