ICL-Router: In-Context Learned Model Representations for LLM Routing
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911265551548416 |
|---|---|
| author | Wang, Chenxu Li, Hao Zhang, Yiqun Chen, Linyao Chen, Jianhao Jian, Ping Ye, Peng Zhang, Qiaosheng Hu, Shuyue |
| author_facet | Wang, Chenxu Li, Hao Zhang, Yiqun Chen, Linyao Chen, Jianhao Jian, Ping Ye, Peng Zhang, Qiaosheng Hu, Shuyue |
| contents | Large language models (LLMs) often exhibit complementary strengths. Model routing harnesses these strengths by dynamically directing each query to the most suitable model, given a candidate model pool. However, routing performance relies on accurate model representations, and adding new models typically requires retraining, limiting scalability. To address these challenges, we propose a novel routing method using in-context vectors to represent model capabilities. The method proceeds in two stages. First, queries are embedded and projected into vectors, with a projector and LLM-based router trained to reconstruct the original queries, aligning vector representations with the router's semantic space. Second, each candidate model is profiled on a query set, and the router learns -- based on in-context vectors of query and model performance -- to predict whether each model can correctly answer new queries. Extensive experiments demonstrate that our method achieves state-of-the-art routing performance in both in-distribution and out-of-distribution tasks. Moreover, our method allows for seamless integration of new models without retraining the router. The code is available at https://github.com/lalalamdbf/ICL-Router. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_09719 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ICL-Router: In-Context Learned Model Representations for LLM Routing Wang, Chenxu Li, Hao Zhang, Yiqun Chen, Linyao Chen, Jianhao Jian, Ping Ye, Peng Zhang, Qiaosheng Hu, Shuyue Machine Learning Artificial Intelligence Large language models (LLMs) often exhibit complementary strengths. Model routing harnesses these strengths by dynamically directing each query to the most suitable model, given a candidate model pool. However, routing performance relies on accurate model representations, and adding new models typically requires retraining, limiting scalability. To address these challenges, we propose a novel routing method using in-context vectors to represent model capabilities. The method proceeds in two stages. First, queries are embedded and projected into vectors, with a projector and LLM-based router trained to reconstruct the original queries, aligning vector representations with the router's semantic space. Second, each candidate model is profiled on a query set, and the router learns -- based on in-context vectors of query and model performance -- to predict whether each model can correctly answer new queries. Extensive experiments demonstrate that our method achieves state-of-the-art routing performance in both in-distribution and out-of-distribution tasks. Moreover, our method allows for seamless integration of new models without retraining the router. The code is available at https://github.com/lalalamdbf/ICL-Router. |
| title | ICL-Router: In-Context Learned Model Representations for LLM Routing |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2510.09719 |