Imagen de Portada

Dual-Pool Token-Budget Routing for Cost-Efficient and Reliable LLM Serving

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Liu, Xunzhuo, He, Bowei, Liu, Xue, Luo, Andy, Zhang, Haichen, Chen, Huamin
Formato:	Preprint
Publicado:	2026
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Token-Budget-Aware Pool Routing for Cost-Efficient LLM Inference
por: Chen, Huamin, et al.
Publicado: (2026)

98$\times$ Faster LLM Routing Without a Dedicated GPU: Flash Attention, Prompt Compression, and Near-Streaming for the vLLM Semantic Router
por: Liu, Xunzhuo, et al.
Publicado: (2026)

Knowledge Access Beats Model Size: Memory Augmented Routing for Persistent AI Agents
por: Liu, Xunzhuo, et al.
Publicado: (2026)

Adaptive Vision-Language Model Routing for Computer Use Agents
por: Liu, Xunzhuo, et al.
Publicado: (2026)

Visual Confused Deputy: Exploiting and Defending Perception Failures in Computer-Using Agents
por: Liu, Xunzhuo, et al.
Publicado: (2026)

Fast and Faithful: Real-Time Verification for Long-Document Retrieval-Augmented Generation Systems
por: Liu, Xunzhuo, et al.
Publicado: (2026)

FleetOpt: Analytical Fleet Provisioning for LLM Inference with Compress-and-Route as Implementation Mechanism
por: Chen, Huamin, et al.
Publicado: (2026)

The 1/W Law: An Analytical Study of Context-Length Routing Topology and GPU Generation Gains for LLM Inference Energy Efficiency
por: Chen, Huamin, et al.
Publicado: (2026)

From Inference Routing to Agent Orchestration: Declarative Policy Compilation with Cross-Layer Verification
por: Chen, Huamin, et al.
Publicado: (2026)

The Workload-Router-Pool Architecture for LLM Inference Optimization: A Vision Paper from the vLLM Semantic Router Project
por: Chen, Huamin, et al.
Publicado: (2026)

inference-fleet-sim: A Queueing-Theory-Grounded Fleet Capacity Planner for LLM Inference
por: Chen, Huamin, et al.
Publicado: (2026)

Outcome-Aware Tool Selection for Semantic Routers: Latency-Constrained Learning Without LLM Inference
por: Chen, Huamin, et al.
Publicado: (2026)

When to Reason: Semantic Router for vLLM
por: Wang, Chen, et al.
Publicado: (2025)

Conflict-Free Policy Languages for Probabilistic ML Predicates: A Framework and Case Study with the Semantic Router DSL
por: Liu, Xunzhuo, et al.
Publicado: (2026)

SelfBudgeter: Adaptive Token Allocation for Efficient LLM Reasoning
por: Li, Zheng, et al.
Publicado: (2025)

Continuous Semantic Caching for Low-Cost LLM Serving
por: Atalar, Baran, et al.
Publicado: (2026)

RouteGoT: Node-Adaptive Routing for Cost-Efficient Graph of Thoughts Reasoning
por: Liu, Yuhang, et al.
Publicado: (2026)

Token-Budget-Aware LLM Reasoning
por: Han, Tingxu, et al.
Publicado: (2024)

OmniRouter: Budget and Performance Controllable Multi-LLM Routing
por: Mei, Kai, et al.
Publicado: (2025)

Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategies
por: Wang, Junlin, et al.
Publicado: (2024)

ParetoBandit: Budget-Paced Adaptive Routing for Non-Stationary LLM Serving
por: Taberner-Miller, Annette
Publicado: (2026)

Coral: Cost-Efficient Multi-LLM Serving over Heterogeneous Cloud GPUs
por: Mei, Yixuan, et al.
Publicado: (2026)

Reasoning Is Not Free: Robust Adaptive Cost-Efficient Routing for LLM-as-a-Judge
por: Zhang, Wenbo, et al.
Publicado: (2026)

DTRNet: Dynamic Token Routing Network to Reduce Quadratic Costs in Transformers
por: Sharma, Aman, et al.
Publicado: (2025)

HeadRouter: Dynamic Head-Weight Routing for Task-Adaptive Audio Token Pruning in Large Audio Language Models
por: He, Peize, et al.
Publicado: (2026)

Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing
por: Ding, Dujian, et al.
Publicado: (2024)

Token-Level LLM Collaboration via FusionRoute
por: Xiong, Nuoya, et al.
Publicado: (2026)

RouteLMT: Learned Sample Routing for Hybrid LLM Translation Deployment
por: Luo, Yingfeng, et al.
Publicado: (2026)

HAMburger: Accelerating LLM Inference via Token Smashing
por: Liu, Jingyu, et al.
Publicado: (2025)

DiffAdapt: Difficulty-Adaptive Reasoning for Token-Efficient LLM Inference
por: Liu, Xiang, et al.
Publicado: (2025)

No Clustering, No Routing: How Transformers Actually Process Rare Tokens
por: Liu, Jing
Publicado: (2025)

Mask Tokens as Prophet: Fine-Grained Cache Eviction for Efficient dLLM Inference
por: Huang, Jianuo, et al.
Publicado: (2025)

Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention
por: Gao, Bin, et al.
Publicado: (2024)

Preserving LLM Capabilities through Calibration Data Curation: From Analysis to Optimization
por: He, Bowei, et al.
Publicado: (2025)

EllieSQL: Cost-Efficient Text-to-SQL with Complexity-Aware Routing
por: Zhu, Yizhang, et al.
Publicado: (2025)

FlexLLM: Token-Level Co-Serving of LLM Inference and Finetuning with SLO Guarantees
por: Oliaro, Gabriele, et al.
Publicado: (2024)

Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognition
por: Chen, Hanting, et al.
Publicado: (2025)

LKV: End-to-End Learning of Head-wise Budgets and Token Selection for LLM KV Cache Eviction
por: Zhou, Enshuai, et al.
Publicado: (2026)

LLM-based Human Simulations Have Not Yet Been Reliable
por: Wang, Qian, et al.
Publicado: (2025)

Rethinking Stepwise Model Routing: A Cost-Efficient Table Reasoning Perspective
por: Ye, Shenghao, et al.
Publicado: (2026)