Image de couverture de livre

KunServe: Parameter-centric Memory Management for Efficient Memory Overloading Handling in LLM Serving

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Cheng, Rongxin, Lai, Yuxin, Wei, Xingda, Chen, Rong, Chen, Haibo
Format:	Preprint
Publié:	2024
Sujets:	Distributed, Parallel, and Cluster Computing Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

ProMoE: Fast MoE-based LLM Serving using Proactive Caching
par: Song, Xiaoniu, et autres
Publié: (2024)

Fast LLM Post-training via Decoupled and Fastest-of-N Speculation
par: Cheng, Rongxin, et autres
Publié: (2025)

DecLock: A Case of Decoupled Locking for Disaggregated Memory
par: Zhang, Hanze, et autres
Publié: (2025)

DiFache: Efficient and Scalable Caching on Disaggregated Memory using Decentralized Coherence
par: Zhang, Hanze, et autres
Publié: (2025)

TinyServe: Query-Aware Cache Selection for Efficient LLM Serving
par: Liu, Dong, et autres
Publié: (2025)

BucketServe: Bucket-Based Dynamic Batching for Smart and Efficient LLM Inference Serving
par: Zheng, Wanyi, et autres
Publié: (2025)

eLLM: Elastic Memory Management Framework for Efficient LLM Serving
par: Xu, Jiale, et autres
Publié: (2025)

Jenga: Effective Memory Management for Serving LLM with Heterogeneity
par: Zhang, Chen, et autres
Publié: (2025)

ScaleSim: Serving Large-Scale Multi-Agent Simulation with Invocation Distance-Based Memory Management
par: Pan, Zaifeng, et autres
Publié: (2026)

Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving
par: Qin, Ruoyu, et autres
Publié: (2024)

VibeServe: Can AI Agents Build Bespoke LLM Serving Systems?
par: Kamahori, Keisuke, et autres
Publié: (2026)

StreamServe: Adaptive Speculative Flows for Low-Latency Disaggregated LLM Serving
par: Kumar, Satyam, et autres
Publié: (2026)

CoServe: Efficient Collaboration-of-Experts (CoE) Model Inference with Limited Memory
par: Suo, Jiashun, et autres
Publié: (2025)

MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool
par: Hu, Cunchen, et autres
Publié: (2024)

Characterizing the Dilemma of Performance and Index Size in Billion-Scale Vector Search and Breaking It with Second-Tier Memory
par: Cheng, Rongxin, et autres
Publié: (2024)

Efficient Serving of LLM Applications with Probabilistic Demand Modeling
par: Liu, Yifei, et autres
Publié: (2025)

FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving
par: Ye, Zihao, et autres
Publié: (2025)

PAM: Processing Across Memory Hierarchy for Efficient KV-centric LLM Serving System
par: Liu, Lian, et autres
Publié: (2026)

Block: Balancing Load in LLM Serving with Context, Knowledge and Predictive Scheduling
par: Da, Wei, et autres
Publié: (2025)

SPECTRE: Hybrid Ordinary-Parallel Speculative Serving for Resource-Efficient LLM Inference
par: Xie, Jincheng, et autres
Publié: (2026)

SkyServe: Serving AI Models across Regions and Clouds with Spot Instances
par: Mao, Ziming, et autres
Publié: (2024)

EPD-Serve: A Flexible Multimodal EPD Disaggregation Inference Serving System On Ascend
par: Bai, Fan, et autres
Publié: (2026)

ENOVA: Autoscaling towards Cost-effective and Stable Serverless LLM Serving
par: Huang, Tao, et autres
Publié: (2024)

PlanetServe: A Decentralized, Scalable, and Privacy-Preserving Overlay for Democratizing Large Language Model Serving
par: Fang, Fei, et autres
Publié: (2025)

LLM Inference Serving: Survey of Recent Advances and Opportunities
par: Li, Baolin, et autres
Publié: (2024)

GhostServe: A Lightweight Checkpointing System in the Shadow for Fault-Tolerant LLM Serving
par: Jayakody, Shakya, et autres
Publié: (2026)

AdaServe: Accelerating Multi-SLO LLM Serving with SLO-Customized Speculative Decoding
par: Li, Zikun, et autres
Publié: (2025)

Equinox: Holistic Fair Scheduling in Serving Large Language Models
par: Wei, Zhixiang, et autres
Publié: (2025)

Autellix: An Efficient Serving Engine for LLM Agents as General Programs
par: Luo, Michael, et autres
Publié: (2025)

LAPS: A Length-Aware-Prefill LLM Serving System
par: She, Jianshu, et autres
Publié: (2026)

Scepsy: Serving Agentic Workflows Using Aggregate LLM Pipelines
par: Wagenländer, Marcel, et autres
Publié: (2026)

PALS: Power-Aware LLM Serving for Mixture-of-Experts Models
par: Hankendi, Can, et autres
Publié: (2026)

ModServe: Modality- and Stage-Aware Resource Disaggregation for Scalable Multimodal Model Serving
par: Qiu, Haoran, et autres
Publié: (2025)

KVCache Cache in the Wild: Characterizing and Optimizing KVCache Cache at a Large Cloud Provider
par: Wang, Jiahao, et autres
Publié: (2025)

DeServe: Towards Affordable Offline LLM Inference via Decentralization
par: Wu, Linyu, et autres
Publié: (2025)

Nightjar: Dynamic Adaptive Speculative Decoding for Large Language Models Serving
par: Li, Rui, et autres
Publié: (2025)

Mell: Memory-Efficient Large Language Model Serving via Multi-GPU KV Cache Management
par: Qianli, Liu, et autres
Publié: (2025)

LR-CNN: Lightweight Row-centric Convolutional Neural Network Training for Memory Reduction
par: Wang, Zhigang, et autres
Publié: (2024)

Beluga: A CXL-Based Memory Architecture for Scalable and Efficient LLM KVCache Management
par: Yang, Xinjun, et autres
Publié: (2025)

KAIROS: Stateful, Context-Aware Power-Efficient Agentic Inference Serving
par: Yuan, Yichao, et autres
Publié: (2026)