Cover Image

SATER: A Self-Aware and Token-Efficient Approach to Routing and Cascading

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shen, Yuanzhe, Liu, Yide, Huang, Zisu, Yin, Ruicheng, Zheng, Xiaoqing, Huang, Xuanjing
Format:	Preprint
Published:	2025
Subjects:	Distributed, Parallel, and Cluster Computing Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Token-Budget-Aware Pool Routing for Cost-Efficient LLM Inference
by: Chen, Huamin, et al.
Published: (2026)

Self-Attention at Constant Cost per Token via Symmetry-Aware Taylor Approximation
by: Heinsen, Franz A., et al.
Published: (2026)

Context Parallelism for Scalable Million-Token Inference
by: Yang, Amy, et al.
Published: (2024)

VoltanaLLM: Feedback-Driven Frequency Control and State-Space Routing for Energy-Efficient LLM Serving
by: Yu, Jiahuan, et al.
Published: (2025)

TACO: Efficient Communication Compression of Intermediate Tensors for Scalable Tensor-Parallel LLM Training
by: Liu, Man, et al.
Published: (2026)

A Fast and Flat Federated Learning Method via Weighted Momentum and Sharpness-Aware Minimization
by: Li, Tianle, et al.
Published: (2025)

SmartMem: Layout Transformation Elimination and Adaptation for Efficient DNN Execution on Mobile
by: Niu, Wei, et al.
Published: (2024)

Cost-Efficient LLM Training with Lifetime-Aware Tensor Offloading via GPUDirect Storage
by: Yuan, Ziqi, et al.
Published: (2025)

Efficient Federated Learning with Timely Update Dissemination
by: Jia, Juncheng, et al.
Published: (2025)

Communication-Efficient Personalized Federal Graph Learning via Low-Rank Decomposition
by: Liu, Ruyue, et al.
Published: (2024)

TinyServe: Query-Aware Cache Selection for Efficient LLM Serving
by: Liu, Dong, et al.
Published: (2025)

Loop Improvement: An Efficient Approach for Extracting Shared Features from Heterogeneous Data without Central Server
by: Li, Fei, et al.
Published: (2024)

ATTENTION2D: Communication Efficient Distributed Self-Attention Mechanism
by: Elango, Venmugil
Published: (2025)

CoLLM: Continuous Adaptation for SLO-Aware LLM Serving on Shared GPU Clusters
by: Huang, Shaoyuan, et al.
Published: (2026)

Sustainable Carbon-Aware and Water-Efficient LLM Scheduling in Geo-Distributed Cloud Datacenters
by: Moore, Hayden, et al.
Published: (2025)

AIConfigurator: Lightning-Fast Configuration Optimization for Multi-Framework LLM Serving
by: Xu, Tianhao, et al.
Published: (2026)

SpecBranch: Speculative Decoding via Hybrid Drafting and Rollback-Aware Branch Parallelism
by: Shen, Yuhao, et al.
Published: (2025)

A-3PO: Accelerating Asynchronous LLM Training with Staleness-aware Proximal Policy Approximation
by: Li, Xiaocan, et al.
Published: (2025)

AB-Training: A Communication-Efficient Approach for Distributed Low-Rank Learning
by: Coquelin, Daniel, et al.
Published: (2024)

ByteScale: Efficient Scaling of LLM Training with a 2048K Context Length on More Than 12,000 GPUs
by: Ge, Hao, et al.
Published: (2025)

From Score Distributions to Balance: Plug-and-Play Mixture-of-Experts Routing
by: Shahout, Rana, et al.
Published: (2025)

SPECTRE: Hybrid Ordinary-Parallel Speculative Serving for Resource-Efficient LLM Inference
by: Xie, Jincheng, et al.
Published: (2026)

FlowSpec: Continuous Pipelined Speculative Decoding for Efficient Distributed LLM Inference
by: Liu, Xing, et al.
Published: (2025)

AdaptiveLoad: Towards Efficient Video Diffusion Transformer Training
by: Guo, Yucheng, et al.
Published: (2026)

PubSub-VFL: Towards Efficient Two-Party Split Learning in Heterogeneous Environments via Publisher/Subscriber Architecture
by: Liu, Yi, et al.
Published: (2025)

MoEBlaze: Breaking the Memory Wall for Efficient MoE Training on Modern GPUs
by: Zhang, Jiyuan, et al.
Published: (2026)

ProTrain: Efficient LLM Training via Memory-Aware Techniques
by: Yang, Hanmei, et al.
Published: (2024)

FedHERO: A Federated Learning Approach for Node Classification Task on Heterophilic Graphs
by: Chen, Zihan, et al.
Published: (2025)

Efficient Training on Multiple Consumer GPUs with RoundPipe
by: Luo, Yibin, et al.
Published: (2026)

Optimizing LLM Inference: Fluid-Guided Online Scheduling with Memory Constraints
by: Ao, Ruicheng, et al.
Published: (2025)

Autellix: An Efficient Serving Engine for LLM Agents as General Programs
by: Luo, Michael, et al.
Published: (2025)

LCFed: An Efficient Clustered Federated Learning Framework for Heterogeneous Data
by: Zhang, Yuxin, et al.
Published: (2025)

D$^{2}$MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Serving
by: Wang, Haodong, et al.
Published: (2025)

LiquidGEMM: Hardware-Efficient W4A8 GEMM Kernel for High-Performance LLM Serving
by: Hu, Huanqi, et al.
Published: (2025)

Federated Neural Architecture Search with Model-Agnostic Meta Learning
by: Huang, Xinyuan, et al.
Published: (2025)

Elastic On-Device LLM Service
by: Yin, Wangsong, et al.
Published: (2024)

Efficient Distributed Retrieval-Augmented Generation for Enhancing Language Model Performance
by: Liu, Shangyu, et al.
Published: (2025)

EdgeLoRA: An Efficient Multi-Tenant LLM Serving System on Edge Devices
by: Shen, Zheyu, et al.
Published: (2025)

Semantic Parallelism: Redefining Efficient MoE Inference via Model-Data Co-Scheduling
by: Li, Yan, et al.
Published: (2025)

TRAIL: Trust-Aware Client Scheduling for Semi-Decentralized Federated Learning
by: Hu, Gangqiang, et al.
Published: (2024)