Image de couverture de livre

InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Lee, Wonbeom, Lee, Jungi, Seo, Junghwan, Sim, Jaewoong
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Distributed, Parallel, and Cluster Computing
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

DiffKV: Differentiated Memory Management for Large Language Models with Parallel KV Compaction
par: Zhang, Yanqi, et autres
Publié: (2024)

KVPR: Efficient LLM Inference with I/O-Aware KV Cache Partial Recomputation
par: Jiang, Chaoyi, et autres
Publié: (2024)

Efficient Remote KV Cache Reuse with GPU-native Video Codec
par: Mi, Liang, et autres
Publié: (2026)

Leyline: KV Cache Directives for Agentic Inference
par: Ma, Bole, et autres
Publié: (2026)

Cost-Efficient LLM Serving in the Cloud: VM Selection with KV Cache Offloading
par: Kim, Kihyun, et autres
Publié: (2025)

FDC: Fast KV Dimensionality Compression for Efficient LLM Inference
par: Zhang, Zeyu, et autres
Publié: (2024)

MoSKA: Mixture of Shared KV Attention for Efficient Long-Sequence LLM Inference
par: Rhee, Myunghyun, et autres
Publié: (2025)

HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference
par: Zhong, Shuzhang, et autres
Publié: (2025)

ForkKV: Scaling Multi-LoRA Agent Serving via Copy-on-Write Disaggregated KV Cache
par: Wang, Shao, et autres
Publié: (2026)

Mell: Memory-Efficient Large Language Model Serving via Multi-GPU KV Cache Management
par: Qianli, Liu, et autres
Publié: (2025)

PolyKV: A Shared Asymmetrically-Compressed KV Cache Pool for Multi-Agent LLM Inference
par: Patel, Ishan, et autres
Publié: (2026)

MorphServe: Efficient and Workload-Aware LLM Serving via Runtime Quantized Layer Swapping and KV Cache Resizing
par: Su, Zhaoyuan, et autres
Publié: (2025)

Enabling Efficient Serverless Inference Serving for LLM (Large Language Model) in the Cloud
par: Ghosh, Himel
Publié: (2024)

PrefillShare: A Shared Prefill Module for KV Reuse in Multi-LLM Disaggregated Serving
par: Woo, Sunghyeon, et autres
Publié: (2026)

ShadowServe: Interference-Free KV Cache Fetching for Distributed Prefix Caching
par: Xiang, Xingyu, et autres
Publié: (2025)

LayerKV: Optimizing Large Language Model Serving with Layer-wise KV Cache Management
par: Xiong, Yi, et autres
Publié: (2024)

InstGenIE: Generative Image Editing Made Efficient with Mask-aware Caching and Scheduling
par: Jiang, Xiaoxiao, et autres
Publié: (2025)

Fast Distributed Inference Serving for Large Language Models
par: Wu, Bingyang, et autres
Publié: (2023)

Efficient Multi-Adapter LLM Serving via Cross-Model KV-Cache Reuse with Activated LoRA
par: Li, Allison, et autres
Publié: (2025)

MatKV: Trading Compute for Flash Storage in LLM Inference
par: Shin, Kun-Woo, et autres
Publié: (2025)

Designing Large Foundation Models for Efficient Training and Inference: A Survey
par: Liu, Dong, et autres
Publié: (2024)

ExeGPT: Constraint-Aware Resource Scheduling for LLM Inference
par: Oh, Hyungjun, et autres
Publié: (2024)

Tender: Accelerating Large Language Models via Tensor Decomposition and Runtime Requantization
par: Lee, Jungi, et autres
Publié: (2024)

Pipette: Automatic Fine-grained Large Language Model Training Configurator for Real-World Clusters
par: Yim, Jinkyu, et autres
Publié: (2024)

ServerlessLLM: Low-Latency Serverless Inference for Large Language Models
par: Fu, Yao, et autres
Publié: (2024)

A Survey on Large Language Model Acceleration based on KV Cache Management
par: Li, Haoyang, et autres
Publié: (2024)

Dynamic Rebatching for Efficient Early-Exit Inference with DREX
par: Liu, Xuting, et autres
Publié: (2025)

SHARe-KAN: Post-Training Vector Quantization for Cache-Resident KAN Inference
par: Smith, Jeff
Publié: (2025)

KV Cache Compression for Inference Efficiency in LLMs: A Review
par: Liu, Yanyu, et autres
Publié: (2025)

Efficient Unified Caching for Accelerating Heterogeneous AI Workloads
par: Wang, Tianze, et autres
Publié: (2025)

InfiniLoRA: Disaggregated Multi-LoRA Serving for Large Language Models
par: Chen, Hongyu, et autres
Publié: (2026)

HACK: Homomorphic Acceleration via Compression of the Key-Value Cache for Disaggregated LLM Inference
par: Zhang, Zeyu, et autres
Publié: (2025)

Efficient Heterogeneous Large Language Model Decoding with Model-Attention Disaggregation
par: Chen, Shaoyuan, et autres
Publié: (2024)

EPIC: Efficient Position-Independent Caching for Serving Large Language Models
par: Hu, Junhao, et autres
Publié: (2024)

SuperFedNAS: Cost-Efficient Federated Neural Architecture Search for On-Device Inference
par: Khare, Alind, et autres
Publié: (2023)

Towards Federated Low-Rank Adaptation of Language Models with Rank Heterogeneity
par: Byun, Yuji, et autres
Publié: (2024)

FineServe: Precision-Aware KV Slab and Two-Level Scheduling for Heterogeneous Precision LLM Serving
par: Bin, Kyungmin, et autres
Publié: (2025)

Efficient LLM Inference with Activation Checkpointing and Hybrid Caching
par: Lee, Sanghyeon, et autres
Publié: (2025)

EARL: Efficient Agentic Reinforcement Learning Systems for Large Language Models
par: Tan, Zheyue, et autres
Publié: (2025)

HexGen: Generative Inference of Large Language Model over Heterogeneous Environment
par: Jiang, Youhe, et autres
Publié: (2023)