Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.09782 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908534783868928 |
|---|---|
| author | Pulishetty, Roshini Ghantasala, Mani Kishan Dasoju, Keerthy Kaushik Mangwani, Niti Garimella, Vishal Mate, Aditya Chatterjee, Somya Kang, Yue Nosakhare, Ehi Hasan, Sadid Srinivasan, Soundar |
| author_facet | Pulishetty, Roshini Ghantasala, Mani Kishan Dasoju, Keerthy Kaushik Mangwani, Niti Garimella, Vishal Mate, Aditya Chatterjee, Somya Kang, Yue Nosakhare, Ehi Hasan, Sadid Srinivasan, Soundar |
| contents | The proliferation of large language models (LLMs) with varying computational costs and performance profiles presents a critical challenge for scalable, cost-effective deployment in real-world applications. We introduce a unified routing framework that leverages a single-head cross-attention mechanism to jointly model query and model embeddings, enabling dynamic selection of the optimal LLM for each input query. Our approach is evaluated on RouterBench, a large-scale, publicly available benchmark encompassing diverse LLM pools and domains. By explicitly capturing fine-grained query-model interactions, our router predicts both response quality and generation cost, achieving up to 6.6% improvement in Average Improvement in Quality (AIQ) and 2.9% in maximum performance over existing routers. To robustly balance performance and cost, we propose an exponential reward function that enhances stability across user preferences. The resulting architecture is lightweight, generalizes effectively across domains, and demonstrates improved efficiency compared to prior methods, establishing a new standard for cost-aware LLM routing. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_09782 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | One Head, Many Models: Cross-Attention Routing for Cost-Aware LLM Selection Pulishetty, Roshini Ghantasala, Mani Kishan Dasoju, Keerthy Kaushik Mangwani, Niti Garimella, Vishal Mate, Aditya Chatterjee, Somya Kang, Yue Nosakhare, Ehi Hasan, Sadid Srinivasan, Soundar Machine Learning The proliferation of large language models (LLMs) with varying computational costs and performance profiles presents a critical challenge for scalable, cost-effective deployment in real-world applications. We introduce a unified routing framework that leverages a single-head cross-attention mechanism to jointly model query and model embeddings, enabling dynamic selection of the optimal LLM for each input query. Our approach is evaluated on RouterBench, a large-scale, publicly available benchmark encompassing diverse LLM pools and domains. By explicitly capturing fine-grained query-model interactions, our router predicts both response quality and generation cost, achieving up to 6.6% improvement in Average Improvement in Quality (AIQ) and 2.9% in maximum performance over existing routers. To robustly balance performance and cost, we propose an exponential reward function that enhances stability across user preferences. The resulting architecture is lightweight, generalizes effectively across domains, and demonstrates improved efficiency compared to prior methods, establishing a new standard for cost-aware LLM routing. |
| title | One Head, Many Models: Cross-Attention Routing for Cost-Aware LLM Selection |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2509.09782 |