ProxRouter: Proximity-Weighted LLM Query Routing for Improved Robustness to Outliers
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918158595522560 |
|---|---|
| author | Patel, Shivam Jali, Neharika Mallick, Ankur Joshi, Gauri |
| author_facet | Patel, Shivam Jali, Neharika Mallick, Ankur Joshi, Gauri |
| contents | Large language model (LLM) query routers are critical to modern AI platforms as they seek to improve efficiency by assigning inference queries to accurate, yet low-cost models. Parametric routers typically use trained neural networks for LLM selection but suffer from retraining and maintenance overheads. Nonparametric routers are training-free, instead estimating LLM accuracy and cost via similarity between encodings of the input query and training set queries. However, like their parametric counterparts, nonparametric routers struggle to generalize to outlier queries, an issue exacerbated by limited diversity in training sets which are costly to expand and difficult to keep current with ever-evolving use cases. We propose ProxRouter, which applies an exponentially tilted aggregation mechanism to balance bias and variance in nonparametric routers, improving their robustness to outliers. Experiments show ProxRouter enhances outlier routing while preserving inlier performance with minimal overhead. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_09852 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ProxRouter: Proximity-Weighted LLM Query Routing for Improved Robustness to Outliers Patel, Shivam Jali, Neharika Mallick, Ankur Joshi, Gauri Machine Learning Artificial Intelligence Large language model (LLM) query routers are critical to modern AI platforms as they seek to improve efficiency by assigning inference queries to accurate, yet low-cost models. Parametric routers typically use trained neural networks for LLM selection but suffer from retraining and maintenance overheads. Nonparametric routers are training-free, instead estimating LLM accuracy and cost via similarity between encodings of the input query and training set queries. However, like their parametric counterparts, nonparametric routers struggle to generalize to outlier queries, an issue exacerbated by limited diversity in training sets which are costly to expand and difficult to keep current with ever-evolving use cases. We propose ProxRouter, which applies an exponentially tilted aggregation mechanism to balance bias and variance in nonparametric routers, improving their robustness to outliers. Experiments show ProxRouter enhances outlier routing while preserving inlier performance with minimal overhead. |
| title | ProxRouter: Proximity-Weighted LLM Query Routing for Improved Robustness to Outliers |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2510.09852 |