Copertina

Mell: Memory-Efficient Large Language Model Serving via Multi-GPU KV Cache Management

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Qianli, Liu, Zicong, Hong, Fahao, Chen, Peng, Li, Song, Guo
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Distributed, Parallel, and Cluster Computing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Communication-Efficient Sparsely-Activated Model Training via Sequence Migration and Token Condensation
di: Chen, Fahao, et al.
Pubblicazione: (2024)

CacheFlow: Efficient LLM Serving with 3D-Parallel KV Cache Restoration
di: Nian, Sean, et al.
Pubblicazione: (2026)

OTAS: An Elastic Transformer Serving System via Token Adaptation
di: Chen, Jinyu, et al.
Pubblicazione: (2024)

TokenDance: Scaling Multi-Agent LLM Serving via Collective KV Cache Sharing
di: Bian, Zhuohang, et al.
Pubblicazione: (2026)

TraCT: Disaggregated LLM Serving with CXL Shared Memory KV Cache at Rack-Scale
di: Yoon, Dongha, et al.
Pubblicazione: (2025)

SPIN: Accelerating Large Language Model Inference with Heterogeneous Speculative Models
di: Chen, Fahao, et al.
Pubblicazione: (2025)

Efficient Remote KV Cache Reuse with GPU-native Video Codec
di: Mi, Liang, et al.
Pubblicazione: (2026)

D$^{2}$MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Serving
di: Wang, Haodong, et al.
Pubblicazione: (2025)

Cost-Efficient LLM Serving in the Cloud: VM Selection with KV Cache Offloading
di: Kim, Kihyun, et al.
Pubblicazione: (2025)

BanaServe: Unified KV Cache and Dynamic Module Migration for Balancing Disaggregated LLM Serving in AI Infrastructure
di: He, Yiyuan, et al.
Pubblicazione: (2025)

A Survey on Large Language Model Acceleration based on KV Cache Management
di: Li, Haoyang, et al.
Pubblicazione: (2024)

Adaptive KV Cache Reuse for Fast Long-Context LLM Serving
di: li, Fei, et al.
Pubblicazione: (2026)

InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management
di: Lee, Wonbeom, et al.
Pubblicazione: (2024)

ForkKV: Scaling Multi-LoRA Agent Serving via Copy-on-Write Disaggregated KV Cache
di: Wang, Shao, et al.
Pubblicazione: (2026)

FairKV: Balancing Per-Head KV Cache for Fast Multi-GPU Inference
di: Zhao, Bingzhe, et al.
Pubblicazione: (2025)

MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool
di: Hu, Cunchen, et al.
Pubblicazione: (2024)

DiffKV: Differentiated Memory Management for Large Language Models with Parallel KV Compaction
di: Zhang, Yanqi, et al.
Pubblicazione: (2024)

eLLM: Elastic Memory Management Framework for Efficient LLM Serving
di: Xu, Jiale, et al.
Pubblicazione: (2025)

MorphServe: Efficient and Workload-Aware LLM Serving via Runtime Quantized Layer Swapping and KV Cache Resizing
di: Su, Zhaoyuan, et al.
Pubblicazione: (2025)

FastCache: Optimizing Multimodal LLM Serving through Lightweight KV-Cache Compression Framework
di: Zhu, Jianian, et al.
Pubblicazione: (2025)

Mélange: Cost Efficient Large Language Model Serving by Exploiting GPU Heterogeneity
di: Griggs, Tyler, et al.
Pubblicazione: (2024)

Federated Fine-Tuning of Sparsely-Activated Large Language Models on Resource-Constrained Devices
di: Chen, Fahao, et al.
Pubblicazione: (2025)

GMLake: Efficient and Transparent GPU Memory Defragmentation for Large-scale DNN Training with Virtual Memory Stitching
di: Guo, Cong, et al.
Pubblicazione: (2024)

DeepServe: Serverless Large Language Model Serving at Scale
di: Hu, Junhao, et al.
Pubblicazione: (2025)

LayerKV: Optimizing Large Language Model Serving with Layer-wise KV Cache Management
di: Xiong, Yi, et al.
Pubblicazione: (2024)

Efficient Multi-Adapter LLM Serving via Cross-Model KV-Cache Reuse with Activated LoRA
di: Li, Allison, et al.
Pubblicazione: (2025)

Serving Chain-structured Jobs with Large Memory Footprints with Application to Large Foundation Model Serving
di: Sun, Tingyang, et al.
Pubblicazione: (2026)

Cache Your Prompt When It's Green: Carbon-Aware Caching for Large Language Model Serving
di: Tian, Yuyang, et al.
Pubblicazione: (2025)

Jenga: Effective Memory Management for Serving LLM with Heterogeneity
di: Zhang, Chen, et al.
Pubblicazione: (2025)

InfiniLoRA: Disaggregated Multi-LoRA Serving for Large Language Models
di: Chen, Hongyu, et al.
Pubblicazione: (2026)

Strata: Hierarchical Context Caching for Long Context Language Model Serving
di: Xie, Zhiqiang, et al.
Pubblicazione: (2025)

Understanding Bottlenecks for Efficiently Serving LLM Inference With KV Offloading
di: Meng, William, et al.
Pubblicazione: (2025)

KunServe: Parameter-centric Memory Management for Efficient Memory Overloading Handling in LLM Serving
di: Cheng, Rongxin, et al.
Pubblicazione: (2024)

PAM: Processing Across Memory Hierarchy for Efficient KV-centric LLM Serving System
di: Liu, Lian, et al.
Pubblicazione: (2026)

DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
di: Zhong, Yinmin, et al.
Pubblicazione: (2024)

BlockLLM: Multi-tenant Finer-grained Serving for Large Language Models
di: Hu, Bodun, et al.
Pubblicazione: (2024)

Cascadia: An Efficient Cascade Serving System for Large Language Models
di: Jiang, Youhe, et al.
Pubblicazione: (2025)

LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelism
di: Wu, Bingyang, et al.
Pubblicazione: (2024)

Memory-Efficient Federated Fine-Tuning of Large Language Models via Layer Pruning
di: Wu, Yebo, et al.
Pubblicazione: (2025)

PackKV: Reducing KV Cache Memory Footprint through LLM-Aware Lossy Compression
di: Jiang, Bo, et al.
Pubblicazione: (2025)