Cover Image

Towards Efficient Key-Value Cache Management for Prefix Prefilling in LLM Inference

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhu, Yue, Yu, Hao, Wang, Chen, Liu, Zhuoran, Lee, Eun Kyung
Format:	Preprint
Published:	2025
Subjects:	Emerging Technologies Artificial Intelligence Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SPAARC: Spatial Proximity and Association based prefetching for Augmented Reality in edge Cache
by: Sreekumar, Nikhil, et al.
Published: (2025)

Smart Space Environments: Key Challenges and Innovative Solutions
by: Kumar, Ramakant
Published: (2024)

Cache Coherence Over Disaggregated Memory
by: Wang, Ruihong, et al.
Published: (2024)

CCCL: Node-Spanning GPU Collectives with CXL Memory Pooling
by: Xu, Dong, et al.
Published: (2026)

Energy-Efficient Split Learning for Resource-Constrained Environments: A Smart Farming Solution
by: Soltani, Keiwan, et al.
Published: (2025)

Native LLM and MLLM Inference at Scale on Apple Silicon
by: Barrios, Wayner
Published: (2026)

Active Inference-Based Adaptive Routing for Heterogeneous Edge AI Services
by: Wang, Zihang, et al.
Published: (2026)

Evaluating Kubernetes Performance for GenAI Inference: From Automatic Speech Recognition to LLM Summarization
by: Malleni, Sai Sindhur, et al.
Published: (2026)

Instant Resonance: Dual Strategy Enhances the Data Consensus Success Rate of Blockchain Threshold Signature Oracles
by: Xian, Youquan, et al.
Published: (2024)

KPI2KVI: A Multi Agent Workflow for Calculating Key Value Indicators from Service Descriptions
by: Shokrnezhad, Masoud, et al.
Published: (2026)

Blockchain Transaction Conflicts: A Historical Perspective
by: Anjana, Parwat Singh, et al.
Published: (2025)

Stencil Computations on Tenstorrent Wormhole
by: Piarulli, Lorenzo, et al.
Published: (2026)

Single Bridge Formation in Self-Organizing Particle Systems
by: Oh, Shunhao, et al.
Published: (2024)

Scaling to 32 GPUs on a Novel Composable System Architecture
by: Ihnotic, John
Published: (2024)

GrapheonRL: A Graph Neural Network and Reinforcement Learning Framework for Constraint and Data-Aware Workflow Mapping and Scheduling in Heterogeneous HPC Systems
by: Sharma, Aasish Kumar, et al.
Published: (2025)

Fork is All You Need in Heterogeneous Systems
by: Wang, Zixuan, et al.
Published: (2024)

Raft Distributed System for Multi-access Edge Computing Sharing Resources
by: Khaliq, Zain, et al.
Published: (2024)

Long-Term or Temporary? Hybrid Worker Recruitment for Mobile Crowd Sensing and Computing
by: Liwang, Minghui, et al.
Published: (2022)

A Scored Non-Deterministic Finite Automata Processor for Sequence Alignment
by: Karakchi, Ryan Karbowniczak Rasha
Published: (2024)

3D System Design: A Case for Building Customized Modular Systems in 3D
by: Emma, Philip, et al.
Published: (2024)

LLMChain: Blockchain-based Reputation System for Sharing and Evaluating Large Language Models
by: Bouchiha, Mouhamed Amine, et al.
Published: (2024)

Quantum Cloud Computing: A Review, Open Problems, and Future Directions
by: Nguyen, Hoa T., et al.
Published: (2024)

Is RISC-V Ready for Machine Learning? Portable Gaussian Processes Using Asynchronous Tasks
by: Strack, Alexander, et al.
Published: (2026)

DRLQ: A Deep Reinforcement Learning-based Task Placement for Quantum Cloud Computing
by: Nguyen, Hoa T., et al.
Published: (2024)

Uniting the World by Dividing it: Federated Maps to Enable Spatial Applications
by: Bharadwaj, Sagar, et al.
Published: (2025)

Parallel FFTW on RISC-V: A Comparative Study including OpenMP, MPI, and HPX
by: Strack, Alexander, et al.
Published: (2025)

Scaling Sample-Based Quantum Diagonalization on GPU-Accelerated Systems using OpenMP Offload
by: Walkup, Robert, et al.
Published: (2026)

Synergistic Integration of Blockchain and Software-Defined Networking in the Internet of Energy Systems
by: Hayyolalam, Vahideh, et al.
Published: (2024)

Resource-Aware Task Allocator Design: Insights and Recommendations for Distributed Satellite Constellations
by: Veeravalli, Bharadwaj
Published: (2026)

Developing a BLAS library for the AMD AI Engine
by: Laan, Tristan, et al.
Published: (2024)

Quantum Cloud Computing: Trends and Challenges
by: Golec, Muhammed, et al.
Published: (2024)

Trustworthy Second-hand Marketplace for Built Environment
by: Wilson, Stanly, et al.
Published: (2025)

Split Federated Learning for Low-Altitude Wireless Networks: Joint Sensing, Communication, Computation, and Control Co-design
by: Hou, Xiangwang, et al.
Published: (2025)

A Programming Model for Disaggregated Memory over CXL
by: Assa, Gal, et al.
Published: (2024)

Enabling SSI-Compliant Use of EUDI Wallet Credentials through Trusted Execution Environment and Zero-Knowledge Proof
by: Sitouah, Nacereddine, et al.
Published: (2026)

A reliability- and latency-driven task allocation framework for workflow applications in the edge-hub-cloud continuum
by: Kouloumpris, Andreas, et al.
Published: (2026)

Investigations of multi-socket high core count RISC-V for HPC workloads
by: Brown, Nick, et al.
Published: (2025)

Distributed Systems in Fintech
by: Mashruwala, Anurag
Published: (2024)

Night-Window Batching versus Carbon-Aware Scheduling for Clinical AI GPU Workloads
by: Doshi, Nishi, et al.
Published: (2026)

Paving the Way to Hybrid Quantum-Classical Scientific Workflows
by: Cranganore, Sandeep Suresh, et al.
Published: (2024)