Rapid AI Gateway Integration LLM API AI Agent LLM API

Facing problems such as multi-model access and lack of unified traffic governance when applying AI technology on a large scale, Ctrip chose to build an AI gateway based on Higress. This solution has the advantages of deep technical accumulation and strong scalability. By uniformly managing large model services and MCP service access, it realizes functions such as authentication, traffic control, and model mapping. The system is currently running stably and supports multiple scenarios. In the future, the ability to convert existing HTTP APIs to MCP services will be optimized.

From 2023 to the present, Junrun HR has begun continuous exploration in the AI ​​field. Among them, how to build digital employees (agents) that can deliver work results for different positions is the focus of exploration. As of now, Junrun has more than 1,000 digital employees working 24/7, saving tens of millions of labor costs cumulatively, and Higress has provided great assistance in this regard.

In the MCP service hosting solution, Higress is responsible for receiving MCP requests and completing protocol offloading, providing unified identity authentication, traffic scheduling, parameter mapping, and security audit capabilities, enabling developers to quickly expose existing services as MCP Servers without in-depth understanding of MCP protocol details.

SOFA AI Gateway is an intelligent gateway solution built on the open-source Higress kernel, deeply optimized for AI business scenarios. It focuses on core functions such as security protection, traffic control, semantic caching, and building financial MCP markets. Through features such as intelligent routing and unified access, this gateway effectively solves challenges such as high latency and high resource consumption in AI scenarios, significantly reducing the complexity of model access and improving the R&D efficiency of agents.

Higress has encapsulated the open-source wasm client, making it easier for developers to develop plugins, such as controlling the scope of plugin effectiveness, and there are already many plugin capabilities in the community that can be used directly. Higress integrates a variety of mainstream registration centers in terms of multi-data sources. These capabilities can be reused directly without repeated construction.

The Higress MCP market provides a complete solution, through which Investoday's rich financial data APIs can be quickly converted into MCP tools. Users can easily call professional financial data through MCP and freely and quickly build their own financial large model applications. As a result, Investoday launched the Investoday Financial Data MCP service, providing one-stop financial data MCP.