INFERENCEDYNAMICS: Efficient Routing Across LLMs through Structured Capability and Knowledge Profiling
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915297908228096 |
|---|---|
| author | Shi, Haochen Zheng, Tianshi Wang, Weiqi Xu, Baixuan Li, Chunyang Chan, Chunkit Fan, Tao Song, Yangqiu Yang, Qiang |
| author_facet | Shi, Haochen Zheng, Tianshi Wang, Weiqi Xu, Baixuan Li, Chunyang Chan, Chunkit Fan, Tao Song, Yangqiu Yang, Qiang |
| contents | Large Language Model (LLM) routing is a pivotal technique for navigating a diverse landscape of LLMs, aiming to select the best-performing LLMs tailored to the domains of user queries, while managing computational resources. However, current routing approaches often face limitations in scalability when dealing with a large pool of specialized LLMs, or in their adaptability to extending model scope and evolving capability domains. To overcome those challenges, we propose InferenceDynamics, a flexible and scalable multi-dimensional routing framework by modeling the capability and knowledge of models. We operate it on our comprehensive dataset RouteMix, and demonstrate its effectiveness and generalizability in group-level routing using modern benchmarks including MMLU-Pro, GPQA, BigGenBench, and LiveBench, showcasing its ability to identify and leverage top-performing models for given tasks, leading to superior outcomes with efficient resource utilization. The broader adoption of Inference Dynamics can empower users to harness the full specialized potential of the LLM ecosystem, and our code will be made publicly available to encourage further research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_16303 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | INFERENCEDYNAMICS: Efficient Routing Across LLMs through Structured Capability and Knowledge Profiling Shi, Haochen Zheng, Tianshi Wang, Weiqi Xu, Baixuan Li, Chunyang Chan, Chunkit Fan, Tao Song, Yangqiu Yang, Qiang Computation and Language Large Language Model (LLM) routing is a pivotal technique for navigating a diverse landscape of LLMs, aiming to select the best-performing LLMs tailored to the domains of user queries, while managing computational resources. However, current routing approaches often face limitations in scalability when dealing with a large pool of specialized LLMs, or in their adaptability to extending model scope and evolving capability domains. To overcome those challenges, we propose InferenceDynamics, a flexible and scalable multi-dimensional routing framework by modeling the capability and knowledge of models. We operate it on our comprehensive dataset RouteMix, and demonstrate its effectiveness and generalizability in group-level routing using modern benchmarks including MMLU-Pro, GPQA, BigGenBench, and LiveBench, showcasing its ability to identify and leverage top-performing models for given tasks, leading to superior outcomes with efficient resource utilization. The broader adoption of Inference Dynamics can empower users to harness the full specialized potential of the LLM ecosystem, and our code will be made publicly available to encourage further research. |
| title | INFERENCEDYNAMICS: Efficient Routing Across LLMs through Structured Capability and Knowledge Profiling |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2505.16303 |