Token
Token
📖 定义
Token is the basic unit for processing text in large language models. Models split input text into a series of tokens for processing, where each token may be a word, subword, or character. Token count directly affects API call costs and response time.
🔗 在 Higress 中的应用
Higress AI Gateway provides token-level traffic management, supporting quota management and rate limiting based on token consumption, helping enterprises precisely control AI call costs.
💡 示例
- 1 "Hello World" in English is typically split into 2 tokens
- 2 Chinese tokenization is usually done by character or word group
- 3 GPT-4's context window supports 128K tokens
⚙️ 配置示例
YAML
# Higress Token Rate Limiting Configuration Example
rule:
limit_by_token:
tokens_per_minute: 100000
tokens_per_day: 1000000🔄 相关术语
❓ 常见问题
Token 是什么？
Token is the basic unit for processing text in large language models. Models split input text into a series of tokens for processing, where each token may be a word, subword, or character. Token count directly affects API call costs and response time.
Higress 如何支持 Token？
Higress AI Gateway provides token-level traffic management, supporting quota management and rate limiting based on token consumption, helping enterprises precisely control AI call costs.
深入了解 Higress
探索更多 Higress 的功能和最佳实践