The rise of Large Language Models (LLMs) and the increasing demands of AI applications mean we're constantly "talking" to AI, providing context for everything from daily tasks to professional work. However, directly copying and pasting raw HTML web content into LLMs can complicate the input and waste valuable tokens, preventing the model from effectively following commands. 1 This clearly highlights the importance of providing LLM-friendly plain text inputs.
Historically, websites primarily served human users and web crawlers. Today, however, websites must also cater to LLMs, enabling them to quickly access content, documentation, and information about product usage. 2 This is precisely why we're introducing /llms.txt
in our Next.js project.
What is /llms.txt
?
Unlike traditional data formats like XML, /llms.txt
isn't about rigid data structures. Its purpose is to be easily readable by language models and agents. Therefore, it adopts a Markdown-like approach to provide clear, plain text. 3
While traditional websites have long used robots.txt
to guide web crawlers on which parts of a site not to access, /llms.txt
serves a complementary yet distinct role: it explicitly provides structured, LLM-friendly content that agents should consume and understand. 4
Here's the specific format for writing your llms.txt
file:
# Title
> Optional description goes here
Optional details go here
## Section name
- [Link title](https://link_url): Optional link details
## Optional
- [Link title](https://link_url)
/llms.txt
with Next.js App Router
Let's use the Next.js v15 App Router as an example to demonstrate how to implement llms.txt
. We'll create an app/llms.txt/route.ts
file and use a GET
RESTful API to serve our content. 5
// app/llms.txt/route.ts
export const dynamic = "force-static"; // or make it dynamic if you want
export function GET() {
return new Response(`# Title
> Optional description goes here
Optional details go here
## Section name
- [Link title](https://link_url): Optional link details
## Optional
- [Link title](https://link_url)
`);
}
Now, you can simply open /llms.txt
in your browser to check the result.
Additionally, if you need to provide a more complex and detailed context for your website, you can create an llms-full.txt
file. This file follows the same format as llms.txt
but allows for more extensive information and context.
Conclusion
The /llms.txt
file offers a straightforward yet powerful method to furnish LLMs with the essential context and information about your website. By adopting this format, you ensure that LLMs can readily access and comprehend your content, thereby enhancing their capacity to assist users and execute tasks efficiently.