Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.00202 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909930171138048 |
|---|---|
| author | Lu, Yifan Liu, Rixin Yuan, Jiayi Cui, Xingqi Zhang, Shenrun Liu, Hongyi Xing, Jiarong |
| author_facet | Lu, Yifan Liu, Rixin Yuan, Jiayi Cui, Xingqi Zhang, Shenrun Liu, Hongyi Xing, Jiarong |
| contents | Today's LLM ecosystem comprises a wide spectrum of models that differ in size, capability, and cost. No single model is optimal for all scenarios; hence, LLM routers have become essential for selecting the most appropriate model under varying circumstances. However, the rapid emergence of various routers makes choosing the right one increasingly challenging. To address this problem, we need a comprehensive router comparison and a standardized leaderboard, similar to those available for models. In this work, we introduce RouterArena, the first open platform enabling comprehensive comparison of LLM routers. RouterArena has (1) a principally constructed dataset with broad knowledge domain coverage, (2) distinguishable difficulty levels for each domain, (3) an extensive list of evaluation metrics, and (4) an automated framework for leaderboard updates. Leveraging our framework, we have produced the initial leaderboard with detailed metrics comparison as shown in Figure 1. Our framework for evaluating new routers is on https://github.com/RouteWorks/RouterArena. Our leaderboard is on https://routeworks.github.io/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_00202 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RouterArena: An Open Platform for Comprehensive Comparison of LLM Routers Lu, Yifan Liu, Rixin Yuan, Jiayi Cui, Xingqi Zhang, Shenrun Liu, Hongyi Xing, Jiarong Machine Learning Today's LLM ecosystem comprises a wide spectrum of models that differ in size, capability, and cost. No single model is optimal for all scenarios; hence, LLM routers have become essential for selecting the most appropriate model under varying circumstances. However, the rapid emergence of various routers makes choosing the right one increasingly challenging. To address this problem, we need a comprehensive router comparison and a standardized leaderboard, similar to those available for models. In this work, we introduce RouterArena, the first open platform enabling comprehensive comparison of LLM routers. RouterArena has (1) a principally constructed dataset with broad knowledge domain coverage, (2) distinguishable difficulty levels for each domain, (3) an extensive list of evaluation metrics, and (4) an automated framework for leaderboard updates. Leveraging our framework, we have produced the initial leaderboard with detailed metrics comparison as shown in Figure 1. Our framework for evaluating new routers is on https://github.com/RouteWorks/RouterArena. Our leaderboard is on https://routeworks.github.io/. |
| title | RouterArena: An Open Platform for Comprehensive Comparison of LLM Routers |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2510.00202 |