llms.txt
llms.txt
Section titled “llms.txt”What is llms.txt
Section titled “What is llms.txt”llms.txt is a standard file format designed for large language models (LLMs). Its goal is to help AI systems understand and access website content more efficiently. Unlike traditional robots.txt (used to restrict crawlers) and sitemap.xml (used for SEO), llms.txt is specifically designed to provide LLMs with a structured and concise navigation of a site’s content.
Why do we need llms.txt
Section titled “Why do we need llms.txt”With the widespread adoption of large language models, website content is becoming an important data source for AI systems. However, the current structure of websites presents several challenges:
- HTML complexity: Web pages contain a large amount of navigation, ads, and styling code, which interferes with LLMs understanding the core content
- Context limitations: LLMs have limited context windows and cannot directly process a full complex web page
- Difficulty locating content: LLMs struggle to quickly locate the most important and relevant content on a site
llms.txt provides a clear and concise Markdown-based index of content, helping LLMs quickly understand a site’s structure and key documents.
llms.txt on the Higress website
Section titled “llms.txt on the Higress website”The Higress official website automatically generates and exposes an llms.txt file that contains the full documentation navigation and links. You can access it at:
This file includes:
- Site overview: The core features and positioning of Higress
- Documentation categories: Organized by Overview, User Guide, Operations Guide, Developer Guide, and other sections
- Complete links: Direct links to all major documentation pages
How to use llms.txt in AI tools
Section titled “How to use llms.txt in AI tools”Typical use cases
Section titled “Typical use cases”- Chatbots / Copilot tools: Before answering questions related to Higress, first read
https://higress.ai/llms.txt, then select 1–3 of the most relevant documentation links based on the query keywords, and fetch those pages as context. - Knowledge-base indexing tools: Treat llms.txt as a “seed index” and only crawl the pages listed in it instead of the entire site, which reduces noise and crawling costs.
- Documentation navigation plugins: In IDEs, browser extensions, or internal assistants, use llms.txt to build a “documentation navigation tree” that lets users or agents jump directly to the corresponding docs.
Working with common AI tools
Section titled “Working with common AI tools”Cursor
Section titled “Cursor”In Cursor, you can use the @Docs feature or add https://higress.ai/llms.txt to the project’s docs configuration. This allows Cursor to prioritize the document structure and links defined in the index file when answering Higress-related questions, resulting in more accurate completions and explanations.
Windsurf
Section titled “Windsurf”In Windsurf, you can reference https://higress.ai/llms.txt directly in a conversation using @https://higress.ai/llms.txt, or configure it in .windsurf/rules so that Windsurf can treat it as a long-term memory source for understanding the Higress documentation structure and gateway configuration examples.
Learn more about Windsurf Memories
Claude Code
Section titled “Claude Code”In Claude Code, you can add https://higress.ai/llms.txt as a docs / knowledge source for your workspace. Claude can then automatically retrieve relevant content from the Higress docs when generating configurations, developing plugins, or analyzing issues.
Learn more about Claude Code docs context configuration
Gemini CLI
Section titled “Gemini CLI”In Gemini CLI, you can pass https://higress.ai/llms.txt via the --context flag or configure it in .gemini/config.json, so that the CLI reads and parses this index file before answering questions related to Higress.
Learn more about Gemini CLI context configuration
In Trae, you can add https://higress.ai/llms.txt as a knowledge source in the project configuration and enable citation in the settings. This allows Trae to automatically reference the official documentation structure when generating or analyzing Higress-related APIs or gateway configurations.
Learn more about Trae knowledge source configuration
In Qoder, you can configure https://higress.ai/llms.txt as an external knowledge file in .qoder/config.yml, or temporarily reference it in a conversation using @docs https://higress.ai/llms.txt. This enables the assistant to automatically consult the documentation index when writing Higress-related code or configurations.
Learn more about Qoder configuration
Other AI tools
Section titled “Other AI tools”For other AI tools that support llms.txt or external documentation indexes, you only need to configure https://higress.ai/llms.txt as a knowledge or documentation entry point. This will help them better understand Higress gateway capabilities and usage when generating or analyzing content.
Automatic generation mechanism
Section titled “Automatic generation mechanism”The llms.txt file of the Higress official website is automatically generated at build time and does not require manual maintenance. The generation process includes:
- Reading the documentation sidebar configuration (
_sidebar.json) - Traversing all documentation directories and files
- Extracting documentation titles and links
- Generating the llms.txt file in the standard format
This ensures that llms.txt always stays in sync with the actual documentation.
Learn more
Section titled “Learn more”- llms.txt proposal: https://llmstxt.org/
- Proposal author: Jeremy Howard (Co-founder of Answer.AI)
- Release date: September 3, 2024