Context Window
Context Window
📖 Definition
Context Window is the maximum token count limit that an LLM can process in a single session. It determines how much conversation history and reference information the model can "remember". Larger context windows support longer conversations and more reference documents.
🔗 How Higress Uses This
Higress can intelligently route to models of different specifications based on request context length, optimizing the balance between cost and performance.
💡 Examples
- 1 GPT-4 Turbo supports a 128K token context window
- 2 Claude 3 supports an ultra-long 200K token context
- 3 Long document analysis requires model support with large context windows
🔄 Related Terms
❓ FAQ
What is Context Window?
Context Window is the maximum token count limit that an LLM can process in a single session. It determines how much conversation history and reference information the model can "remember". Larger context windows support longer conversations and more reference documents.
How does Higress support Context Window?
Higress can intelligently route to models of different specifications based on request context length, optimizing the balance between cost and performance.
Learn More About Higress
Explore more Higress features and best practices