# llms.txt Implementation Services in 2026: What to Buy **URL:** https://mettevo.com/blog/article/llmstxt-implementation **Published:** 2026-11-06 **Updated:** 2026-11-06 **Author:** Oleg Silin **Category:** Blog | Mettevo > Implement llms.txt with Mettevo Digital Agency to help AI systems understand your website content. Improve content accessibility and strengthen your website's AI search visibility in 2026. ![llms.txt Implementation](https://stage.mettevo.com/wp-content/uploads/2026/11/Team_giving_document_to_robot_e9a1559c.jpg) --- llms.txt implementation is writing, serving and maintaining a Markdown file at /llms.txt that lists your canonical pages with short descriptions for AI (artificial intelligence) systems. [Google's guide](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide), updated July 10, 2026, says Google Search ignores the file, and neither OpenAI's nor Anthropic's crawler documentation says its crawlers read it. Pay for generation and upkeep, never for guaranteed citations. ## **TL;DR** - llms.txt is a proposed Markdown file at a site's root that lists curated, described pages for AI systems. The proposal at [llmstxt.org](https://llmstxt.org/) is dated September 3, 2024, and its current text is v2 (August 2026). - Google Search ignores llms.txt, according to [Google's guide](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide) (July 10, 2026). The OpenAI and Anthropic crawler documents we read on October 29, 2026 do not list it as a signal. - llms.txt does not control access, guarantee citations, replace robots.txt or block training. - Pay for rules, generation and upkeep, not for guaranteed citations, daily audits with no method or pricing by file size. ## **What is llms.txt, and what does it do?** llms.txt is a proposed Markdown file, served at a site's root, that gives AI systems a curated list of the pages that best explain the site. Jeremy Howard's proposal at [llmstxt.org](https://llmstxt.org/) carries the date September 3, 2024. Each entry has a title, a URL and, optionally, a short note (the spec calls for "brief, informative descriptions" without a fixed length). Per the spec, only an H1 with the site name is required; the summary, free text and H2 link sections are optional. Version 2 also defines path-level files: a file covers the pages under its path, and the most specific one applies. The stated aim is inference, not training. The proposal says it expected the file to be mainly useful "for inference rather than training", "and that is how it has been used". That is the proposal's own account of use. ### **Is llms.txt the same as robots.txt or sitemap.xml?** No. In llms.txt vs robots.txt, the difference is context versus access: llmstxt.org says robots.txt "lets automated tools know what access to a site is considered acceptable", while the file supplies context. Against sitemaps, it says "sitemaps list all pages for search engines" and llms.txt "offers a curated overview for LLMs". ## **Which AI systems actually read llms.txt in 2026?** None of the Google, OpenAI or Anthropic documents we read says the vendor's crawlers fetch other sites' llms.txt files. We sorted the evidence into verifiable, claimed and undocumented, as of October 29, 2026. System What its documents say Status Google Search The guide, updated July 10, 2026, says "Google Search ignores them".  Verifiable: not used. OpenAI crawlers The crawler page names OAI-SearchBot, GPTBot and ChatGPT-User. It mentions llms.txt only as its own docs index. Undocumented as a signal. Anthropic crawlers and web fetch tool The help article (April 7, 2026) never mentions llms.txt. The tool fetches only URLs already in the conversation. Undocumented as a signal. Agents pointed at a file The proposal says agents "are expected to view or search llms.txt". Claimed by the proposal. Publishing is not reading. It is verifiable that OpenAI and Anthropic each serve an llms.txt for their developer docs. The proposal counts such files as adoption, along with "thousands of sites" and Chrome's Lighthouse, which "audits sites for one as part of its agentic browsing checks". An audit shows that a file exists, not that a model used it. Developer documentation is the strongest case, since someone can point a coding agent at the file. For a marketing site, we found no vendor commitment. Limits: three vendors, public documents, one reading on October 29, 2026. Undocumented means unpromised, not unused. We analyzed no server logs, so we cannot say whether any AI system fetches our own file. OpenAI's publisher FAQ refused our fetch tool (403 error), so it is outside this review. ## **The three levels of llms.txt implementation** Level What it produces Where it fits How it fails One: static file A hand-written Markdown file at /llms.txt. A site with a few dozen stable key pages. Links rot as pages move. Two: generated from the CMS (content management system) A file built from CMS records by inclusion rules and rebuilt on publish. A growing site with several sections or languages. It lists every page, or a template error serves a broken file. Three: llms-full.txt A second file that carries page text instead of links. Documentation that agents read in bulk. It grows too large, goes stale or republishes text you did not mean to share. Level three is a convention, not part of the specification: the v2 text and its [Changes page](https://llmstxt.org/changes.html) do not use the name llms-full.txt (we searched both on October 29, 2026). OpenAI's [llms-full.txt](https://platform.openai.com/docs/llms-full.txt) calls itself a "Single-file Markdown export", and our fetch tool measured it at about 5.7 million characters.  ### **How do you implement llms.txt yourself?** 1. List the pages you would send a new customer, each with a recognizable title and one factual sentence. 2. Keep canonical URLs only, each returning a 200 status without a redirect and allowed by robots.txt. 3. Write the file: an H1, a blockquote summary, then H2 sections of - [Title](https://mettevo.com/URL): description lines. 4. Serve it at /llms.txt on your own domain, request the URL to confirm a 200 status, then commit it to version control with a dated change log. The spec's only requirement is an H1 with the site name. For a minimal useful form (one that gives agents enough to work with), start from this file for a fictional company: \# Example Co \> Payroll software for small agencies. This file lists the pages that explain the product, pricing and support. \## Products \- \[Payroll for agencies\](https://example.com/payroll): What the product does, who it fits and which countries it supports. \- \[Pricing\](https://example.com/pricing): How plans are structured and what each plan includes. \## Support \- \[Help center\](https://example.com/help): Setup guides and answers to billing questions. \## Optional \- \[Changelog\](https://example.com/changelog): Dated product updates, secondary to the pages above. ## **What a real llms.txt implementation service delivers** A real service delivers five things. Each comes with an artifact you can open before you pay for the next stage. 1. **File structure review.** The file is checked against the v2 text: H1, summary, section logic and whether a path-level file is warranted. Artifact: an annotated copy. 2. **Canonical URL selection.** Written rules decide which URLs enter the file: indexable, canonical, a 200 status without a redirect, allowed by robots.txt. Our guide to [canonical tags](https://mettevo.com/blog/article/canonical-tag-syntax-examples-and-search-console-errors) covers the mechanism. Artifact: inclusion and exclusion lists with reasons. 3. **Title and description discipline.** Each entry gets a specific name and one factual sentence, with no keyword stuffing and no adjectives a model cannot verify. It is [on-page SEO](https://mettevo.com/seo/on-page-seo) craft. Artifact: a style rule and the final copy. 4. **A programmatic variant for growing sites.** A generator reads CMS records, applies the inclusion rules and rebuilds the file on publish, with a staging copy you can read first. Artifact: the code, handed over. 5. **A versioning plan.** The file lives in version control with a dated change log and a rollback path, and it is reviewed when URLs move or the spec changes. Artifact: the repository and change log. ### **What does a file structure review find in practice?** We ran the review on our own file, mettevo.com/llms.txt, on October 29, 2026. It has an H1, a blockquote summary and 8 H2 sections holding 110 entries. We found three things: - Of 110 total entries, 44 carry a one-line description. The Team section (22 of 22) and the Case Studies section (20 of 21) are the described groups, with two Engineering Guides entries. The All Guides, All Projects and Blog Index links, the XML Sitemap link and the 42 Services, 10 blog article, 4 Company and 6 Locations entries are bare. - The file has no date or version marker. - Our Markdown renditions sit at /md paths, while the v2 text names page.html.md and page.md. None of this makes the file invalid under v2, where only the H1 is required. The counts come from a page-fetch tool for one site on one day, and we did not check for the rel="alternate" link that v2 recommends. To repeat the count, tally the lines that begin with - \[. ### **How much does llms.txt implementation cost, and what drives it?** This article publishes no prices. The shape of the bill is more useful: a one-time setup for the file, its rules and the review, plus an optional ongoing cost for upkeep. Four drivers move the cost: - **Site size.** How many indexable URLs need a decision and a description. - **CMS complexity.** Whether titles, summaries and canonical URLs sit in structured fields. - **Programmatic requirement.** A generator with tests and caching costs more than a static file. - **Update cadence.** Daily publishing needs automation and monitoring; two changes a year need a reminder. ## **What llms.txt does not do** - **llms.txt does not control crawler access.** robots.txt does, and Google adds that robots.txt is "not a mechanism for keeping a web page out of Google". - **llms.txt does not guarantee citation.** The proposal makes no promise that any system reads the file, and Google Search ignores it. OpenAI says sites that opt out of OAI-SearchBot do not appear in ChatGPT search answers, though they can still appear as navigational links, so crawler access matters more than any file. - **llms.txt does not replace robots.txt.** The proposal says the file "can complement robots.txt by providing context for allowed content". A URL listed in llms.txt but disallowed in robots.txt sends mixed signals, so keep the two consistent. - **llms.txt does not block training.** OpenAI, Anthropic and Google each document robots.txt controls for training use (GPTBot, ClaudeBot, Google-Extended), and the proposal says "training runs could take advantage of the information too". ## **How llms.txt, robots.txt, sitemap.xml and schema.org work together in 2026** Signal Question it answers What the source says robots.txt Which URLs may a crawler fetch? Google: it tells crawlers which URLs they can access. sitemap.xml Which URLs exist on the site? Google: search engines read it to crawl more efficiently, with no guarantee of crawling or indexing. Structured data (schema.org) What is on this page? Google: it uses structured data to understand a page, and its AI features need no special schema.org markup. llms.txt Which pages best explain this site? The proposal: a curated overview for LLMs. Google Search ignores it. On mettevo.com, as of October 29, 2026, robots.txt carries a Sitemap line and names 16 AI user agents in one group. The sitemap holds 447 URLs in our count, 288 of them blog articles. llms.txt holds 110 entries and links to the sitemap. Our rule: every URL in llms.txt should be canonical, return a 200 status without a redirect, appear in sitemap.xml and be allowed by robots.txt. We have not run that check across all 110 of our entries, so we do not claim our file passes it. ## **The 5 questions to ask any llms.txt implementation service before signing** For the wider agency checklist, see [how to choose an AI SEO agency](https://mettevo.com/blog/article/how-to-choose-an-ai-seo-agency). 1. **Which URLs go in the file, and who decides?** Good: written inclusion rules and an exclusion list with reasons. Weak: "all your pages". 2. **How is the file produced and kept current?** Good: a named source of truth in your CMS and a rebuild on publish. Weak: a quarterly manual edit for a site that publishes weekly. 3. **What is the versioning plan?** Good: version control, a dated change log, a review when the spec or your URLs change, and a rollback path. Weak: "we overwrite the file". 4. **How will you measure whether anything reads it?** Good: a server-log filter for /llms.txt requests by user agent over a stated window, noting that a fetch is not a citation. Weak: a score with no stated method. 5. **What do I own if we stop?** Good: the file, the generator code, the rules and the change log, served from your domain. Weak: a file hosted on the provider's server and redirected from yours. ## **Red flags in an llms.txt proposal** Check four red flags before you read the price. Mettevo sells [SEO services](https://mettevo.com/seo) and GEO (generative engine optimization) work, so apply them to us first. - **"Guaranteed AI citations."** Google's [hiring guidance](https://developers.google.com/search/docs/fundamentals/do-i-need-seo) (updated June 5, 2026) says "No one can guarantee a #1 ranking on Google". For llms.txt the case is weaker still, because the proposal promises no reader and Google Search ignores the file. I would not sign a contract that promises a citation. - **"Daily llms.txt audits" with no measurement method.** Ask what the audit measures. A daily check that every listed URL returns a 200 status is a short script, not a service. - **No versioning plan.** The spec itself changed in August 2026, when v2 defined path-level files, dropped a context-expansion tool and changed what "Optional" means, per its Changes page. A file nobody versions cannot show what changed or roll back a bad edit. - **Charging by file size or per file.** The effort sits in inclusion rules, descriptions, generation and upkeep, not in line count. Per-line pricing rewards a longer file, while the spec starts from the point that context windows "are still too small for most websites in their entirety". ## **FAQ** ### **Do I need llms.txt for my site?** No. Google says you do not need AI text files, such as llms.txt, to appear in Google Search, and the proposal describes a convention, not a requirement. Publish one if agents or developers are likely to be pointed at your pages, or if a curated page list helps your team. Skip a paid monitoring contract until server logs show requests for it. ### **Does Google read llms.txt?** Google Search does not. Google's guide to optimizing for generative AI features, last updated July 10, 2026, says Google Search ignores llms.txt files and that keeping one will neither harm nor help visibility or rankings there. The guide speaks to Google Search; it does not cover other Google products or other companies' systems. ### **How much does llms.txt implementation cost?** Cost follows four drivers: the number of indexable URLs, how the CMS stores titles and summaries, whether code must generate the file, and how often pages change. A static file is a one-time setup. A generated file with upkeep adds an ongoing cost for monitoring, rebuilds and audits. This article lists no figures because a number without scope misleads. ### **Should we implement llms.txt in-house or hire a service?** Build a static file in-house if a developer can add a file to your web root and your key pages rarely move. Hire help when at least two of these hold: you publish weekly, URLs sit in a CMS with redirects, you need Markdown renditions or an llms-full.txt export, or nobody owns link health. That threshold is our rule of thumb, not a benchmark. Not sure whether llms.txt is worth paying for on your site? Talk to [Mettevo](https://mettevo.com/contacts) and we will tell you honestly