Image de couverture de livre

Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Gao, Shihong, Zhang, Xin, Shen, Yanyan, Chen, Lei
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

RelServe: Fast LLM Inference Serving on Relational Data
par: Zhang, Xin, et autres
Publié: (2025)

SCORPIO: Serving the Right Requests at the Right Time for Heterogeneous SLOs in LLM Inference
par: Tang, Yinghao, et autres
Publié: (2025)

Sparse Prefix Caching for Hybrid and Recurrent LLM Serving
par: Shirokikh, Mikhail, et autres
Publié: (2026)

Joint Encoding of KV-Cache Blocks for Scalable LLM Serving
par: Kampeas, Joseph, et autres
Publié: (2026)

TokenFlow: Responsive LLM Text Streaming Serving under Request Burst via Preemptive Scheduling
par: Chen, Junyi, et autres
Publié: (2025)

DuetServe: Harmonizing Prefill and Decode for LLM Serving via Adaptive GPU Multiplexing
par: Gao, Lei, et autres
Publié: (2025)

MEPIC: Memory Efficient Position Independent Caching for LLM Serving
par: Wang, Qian, et autres
Publié: (2025)

Continuous Semantic Caching for Low-Cost LLM Serving
par: Atalar, Baran, et autres
Publié: (2026)

Lethe: Layer- and Time-Adaptive KV Cache Pruning for Reasoning-Intensive LLM Serving
par: Zeng, Hui, et autres
Publié: (2025)

Collaborative Speculative Inference for Efficient LLM Inference Serving
par: Gao, Luyao, et autres
Publié: (2025)

MorphServe: Efficient and Workload-Aware LLM Serving via Runtime Quantized Layer Swapping and KV Cache Resizing
par: Su, Zhaoyuan, et autres
Publié: (2025)

JITServe: SLO-aware LLM Serving with Imprecise Request Information
par: Zhang, Wei, et autres
Publié: (2025)

AugServe: Adaptive Request Scheduling for Augmented Large Language Model Inference Serving
par: Wang, Ying, et autres
Publié: (2025)

EVICPRESS: Joint KV-Cache Compression and Eviction for Efficient LLM Serving
par: Feng, Shaoting, et autres
Publié: (2025)

Prompt-Aware Scheduling for Low-Latency LLM Serving
par: Tao, Yiheng, et autres
Publié: (2025)

Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantization
par: Kim, Minsu, et autres
Publié: (2025)

Locality-aware Fair Scheduling in LLM Serving
par: Cao, Shiyi, et autres
Publié: (2025)

Unifying Sparse Attention with Hierarchical Memory for Scalable Long-Context LLM Serving
par: Zhao, Zihan, et autres
Publié: (2026)

Rethinking Caching for LLM Serving Systems: Beyond Traditional Heuristics
par: Kim, Jungwoo, et autres
Publié: (2025)

CascadeServe: Unlocking Model Cascades for Inference Serving
par: Kossmann, Ferdi, et autres
Publié: (2024)

Preble: Efficient Distributed Prompt Scheduling for LLM Serving
par: Srivatsa, Vikranth, et autres
Publié: (2024)

ReasonCache: Accelerating Large Reasoning Model Serving through KV Cache Sharing
par: Chen, Kaiwen, et autres
Publié: (2025)

IC-Cache: Efficient Large Language Model Serving via In-context Caching
par: Yu, Yifan, et autres
Publié: (2025)

RW-TTT: Batched Serving for Request-Owned Test-Time Training State
par: Yang, Jian, et autres
Publié: (2026)

MTServe: Efficient Serving for Generative Recommendation Models with Hierarchical Caches
par: Wang, Xin, et autres
Publié: (2026)

PRISM: Fast Online LLM Serving via Scheduling-Memory Co-design
par: Qu, Xingyu, et autres
Publié: (2026)

Semantic Caching for Low-Cost LLM Serving: From Offline Learning to Online Adaptation
par: Liu, Xutong, et autres
Publié: (2025)

HELIOS: Adaptive Model And Early-Exit Selection for Efficient LLM Inference Serving
par: Kumar, Avinash, et autres
Publié: (2025)

Adaptive Request Scheduling for CodeLLM Serving with SLA Guarantees
par: Chang, Shi, et autres
Publié: (2025)

WarmServe: Enabling One-for-Many GPU Prewarming for Multi-LLM Serving
par: Lou, Chiheng, et autres
Publié: (2025)

Fast Heterogeneous Serving: Scalable Mixed-Scale LLM Allocation for SLO-Constrained Inference
par: Cheng, Jiaming, et autres
Publié: (2026)

FineServe: Precision-Aware KV Slab and Two-Level Scheduling for Heterogeneous Precision LLM Serving
par: Bin, Kyungmin, et autres
Publié: (2025)

CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion
par: Yao, Jiayi, et autres
Publié: (2024)

Rethinking Key-Value Cache Compression Techniques for Large Language Model Serving
par: Gao, Wei, et autres
Publié: (2025)

Scalable AI Inference: Performance Analysis and Optimization of AI Model Serving
par: Pham, Hung Cuong, et autres
Publié: (2026)

Niyama : Breaking the Silos of LLM Inference Serving
par: Goel, Kanishk, et autres
Publié: (2025)

On Evaluating Performance of LLM Inference Serving Systems
par: Agrawal, Amey, et autres
Publié: (2025)

HyGen: Efficient LLM Serving via Elastic Online-Offline Request Co-location
par: Sun, Ting, et autres
Publié: (2025)

HybridServe: Efficient Serving of Large AI Models with Confidence-Based Cascade Routing
par: Xue, Leyang, et autres
Publié: (2025)

POLAR: Online Learning for LoRA Adapter Caching and Routing in Edge LLM Serving
par: Li, Shaoang, et autres
Publié: (2026)