llms.txt: The File That Quickly Became Relevant

Published in September 2024 by Jeremy Howard, llms.txt is a proposed standard for a lightweight file that tells LLMs what a website is about and which pages to prioritize.

While the idea is appealing and the effort is minimal, adoption has been spotty, probably because the benefits of an llms.txt file haven’t been proven yet.

So what has changed this spring? On May 5, 2026, Google added llms.txt to its Chrome Lighthouse toolset, under Agentic Browser Audits.

Two weeks later, during the I/O 2026 keynote, Google announced a shift to Agentic Search and the biggest changes to its search experience in decades.

What does it mean for enterprise publishers?

What Google Is Saying

In its documentation, Google mentions llms.txt but seems to contradict itself.

On Google Search Central, under Mythbusting generative AI search: what you don’t need to do, we can read:

LLMS.txt files and other “special” markup: You don’t need to create new machine readable files, AI text files, markup, or Markdown to appear in generative AI search.

Loud and clear. Why waste our time? Well, let’s look at what Google says about the same file in its Lighthouse documentation:

The llms.txt file is an emerging convention used to provide a machine-readable summary of a website’s content, specifically designed for LLMs and AI agents.

Without this file, agents may spend more time crawling the site to understand its high-level structure and primary content.

At a surface level, this sounds like a contradiction, but is it?

RELATED ARTICLE: LLMs.txt Shows No Clear Effect On AI Citations, Based On 300k Domains

The Added Layer of Agentic Search

While the Google Search Central claim has been true for more than a year, a lot—if not everything—is changing with Agentic Search.

Answer Engine Optimization (AEO) results might not be influenced by the presence of an llms.txt file, but now that agents are going to be actively involved in search and monitoring on behalf of their users, llms.txt will be relevant for Agentic Search Engine Optimization (ASEO.)

I Already Have a Sitemap. Is a llms.txt File Worth Adding?

You should have a sitemap, and yes at this point, adding an llms.txt file seems worth it. The official documentation states that:

llms.txt is designed to coexist with current web standards. While sitemaps list all pages for search engines, llms.txt offers a curated overview for LLMs. It can complement robots.txt by providing context for allowed content. The file can also reference structured data markup used on the site, helping LLMs understand how to interpret this information in context.

In other words:

• Site map = all indexable pages
• llms.txt = your site’s elevator pitch for agents

Worth the Low Level of Effort

While llms.txt probably won’t meaningfully improve AI Overviews / generative search visibility on its own, it may become useful in the very near future.

Given the low effort, adding an llms.txt file seems like a reasonable “cheap insurance” move.

The format is intentionally simple. At minimum, it includes a brief description of your site, and optionally, links to your most important pages or sections along with short notes on what each covers. While the file has a .txt extension, the contents are written in Markdown.

Just like robots.txt, llms.txt lives under the root of your site. Some hosts make it hard to access this critical part of your server, but we have a lightweight custom plugin that lets you create and update your llms.txt directly from the WordPress admin. Let us know if we can help.