RouteLLM: Learning to Route LLMs with Preference Data
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909504837255168 |
|---|---|
| author | Ong, Isaac Almahairi, Amjad Wu, Vincent Chiang, Wei-Lin Wu, Tianhao Gonzalez, Joseph E. Kadous, M Waleed Stoica, Ion |
| author_facet | Ong, Isaac Almahairi, Amjad Wu, Vincent Chiang, Wei-Lin Wu, Tianhao Gonzalez, Joseph E. Kadous, M Waleed Stoica, Ion |
| contents | Large language models (LLMs) exhibit impressive capabilities across a wide range of tasks, yet the choice of which model to use often involves a trade-off between performance and cost. More powerful models, though effective, come with higher expenses, while less capable models are more cost-effective. To address this dilemma, we propose several efficient router models that dynamically select between a stronger and a weaker LLM during inference, aiming to optimize the balance between cost and response quality. We develop a training framework for these routers leveraging human preference data and data augmentation techniques to enhance performance. Our evaluation on widely-recognized benchmarks shows that our approach significantly reduces costs-by over 2 times in certain cases-without compromising the quality of responses. Interestingly, our router models also demonstrate significant transfer learning capabilities, maintaining their performance even when the strong and weak models are changed at test time. This highlights the potential of these routers to provide a cost-effective yet high-performance solution for deploying LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_18665 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | RouteLLM: Learning to Route LLMs with Preference Data Ong, Isaac Almahairi, Amjad Wu, Vincent Chiang, Wei-Lin Wu, Tianhao Gonzalez, Joseph E. Kadous, M Waleed Stoica, Ion Machine Learning Artificial Intelligence Computation and Language Large language models (LLMs) exhibit impressive capabilities across a wide range of tasks, yet the choice of which model to use often involves a trade-off between performance and cost. More powerful models, though effective, come with higher expenses, while less capable models are more cost-effective. To address this dilemma, we propose several efficient router models that dynamically select between a stronger and a weaker LLM during inference, aiming to optimize the balance between cost and response quality. We develop a training framework for these routers leveraging human preference data and data augmentation techniques to enhance performance. Our evaluation on widely-recognized benchmarks shows that our approach significantly reduces costs-by over 2 times in certain cases-without compromising the quality of responses. Interestingly, our router models also demonstrate significant transfer learning capabilities, maintaining their performance even when the strong and weak models are changed at test time. This highlights the potential of these routers to provide a cost-effective yet high-performance solution for deploying LLMs. |
| title | RouteLLM: Learning to Route LLMs with Preference Data |
| topic | Machine Learning Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2406.18665 |