Imagen de Portada

Accelerating LLM Inference via Dynamic KV Cache Placement in Heterogeneous Memory System

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Fang, Yunhua, Xie, Rui, Haq, Asad Ul, Ma, Linsen, Maghraoui, Kaoutar El, Wang, Naigang, Wang, Meng, Liu, Liu, Zhang, Tong
Formato:	Preprint
Publicado:	2025
Materias:	Hardware Architecture Artificial Intelligence Performance
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Reimagining Memory Access for LLM Inference: Compression-Aware Memory Controller Design
por: Xie, Rui, et al.
Publicado: (2025)

Making Strong Error-Correcting Codes Work Effectively for HBM in AI Inference
por: Xie, Rui, et al.
Publicado: (2025)

Breaking the HBM Bit Cost Barrier: Domain-Specific ECC for AI Inference Infrastructure
por: Xie, Rui, et al.
Publicado: (2025)

TRACE: Unlocking Effective CXL Bandwidth via Lossless Compression and Precision Scaling
por: Xie, Rui, et al.
Publicado: (2025)

Adaptive Cache Pollution Control for Large Language Model Inference Workloads Using Temporal CNN-Based Prediction and Priority-Aware Replacement
por: Liu, Songze, et al.
Publicado: (2025)

Heterogeneous Memory Benchmarking Toolkit
por: Ghaemi, Golsana, et al.
Publicado: (2025)

3RSeT: Read Disturbance Rate Reduction in STT-MRAM Caches by Selective Tag Comparison
por: Cheshmikhani, Elham, et al.
Publicado: (2025)

Enhancing Instruction Prefetching via Cache and TLB Management
por: Jamet, Alexandre Valentin, et al.
Publicado: (2026)

The Bicameral Cache: a split cache for vector architectures
por: Rebolledo, Susana, et al.
Publicado: (2024)

Gem5-AcceSys: Enabling System-Level Exploration of Standard Interconnects for Novel Accelerators
por: Liu, Qunyou, et al.
Publicado: (2025)

OPTIMA: Design-Space Exploration of Discharge-Based In-SRAM Computing: Quantifying Energy-Accuracy Trade-Offs
por: Seyedfaraji, Saeed, et al.
Publicado: (2024)

ARKV: Adaptive and Resource-Efficient KV Cache Management under Limited Memory Budget for Long-Context Inference in LLMs
por: Lei, Jianlong, et al.
Publicado: (2026)

ChatNeuroSim: An LLM Agent Framework for Automated Compute-in-Memory Accelerator Deployment and Optimization
por: Lee, Ming-Yen, et al.
Publicado: (2026)

Examem: Low-Overhead Memory Instrumentation for Intelligent Memory Systems
por: Poduval, Ashwin, et al.
Publicado: (2024)

A Quantitative Analysis and Guidelines of Data Streaming Accelerator in Modern Intel Xeon Scalable Processors
por: Kuper, Reese, et al.
Publicado: (2023)

SAHM: State-Aware Heterogeneous Multicore for Single-Thread Performance
por: Wadle, Shayne, et al.
Publicado: (2025)

A Review on Proprietary Accelerators for Large Language Models
por: Park, Sihyeong, et al.
Publicado: (2025)

Non-Monotonic Latency in Apple MPS Decoding: KV Cache Interactions and Execution Regimes
por: Hendria, Willy Fitra
Publicado: (2026)

ETM2: Empowering Traditional Memory Bandwidth Regulation using ETM
por: Zuepke, Alexander, et al.
Publicado: (2026)

A$^3$PIM: An Automated, Analytic and Accurate Processing-in-Memory Offloader
por: Jiang, Qingcai, et al.
Publicado: (2024)

Improving the Serving Performance of Multi-LoRA Large Language Models via Efficient LoRA and KV Cache Management
por: Zhang, Hang, et al.
Publicado: (2025)

An Analytical Cost Model for Fast Evaluation of Multiple Compute-Engine CNN Accelerators
por: Qararyah, Fareed, et al.
Publicado: (2025)

BitDecoding: Unlocking Tensor Cores for Long-Context LLMs with Low-Bit KV Cache
por: Du, Dayou, et al.
Publicado: (2025)

Random Adaptive Cache Placement Policy
por: Ahire, Vrushank, et al.
Publicado: (2025)

Accelerating Transistor-Level Simulation of Integrated Circuits via Equivalence of RC Long-Chain Structures
por: Tang, Ruibai, et al.
Publicado: (2025)

Towards CPU Performance Prediction: New Challenge Benchmark Dataset and Novel Approach
por: Liu, Xiaoman
Publicado: (2024)

Context-Aware Mixture-of-Experts Inference on CXL-Enabled GPU-NDP Systems
por: Fan, Zehao, et al.
Publicado: (2025)

VeriCache: Turning Lossy KV Cache into Lossless LLM Inference
por: Yao, Jiayi, et al.
Publicado: (2026)

LLM-Driven Design Space Exploration of FPGA-based Accelerators
por: Sharma, Vinamra, et al.
Publicado: (2026)

SCALE-Sim TPU: Validating and Extending SCALE-Sim for TPUs
por: Dang, Jingtian, et al.
Publicado: (2026)

SCALE-Sim v3: A modular cycle-accurate systolic accelerator simulator for end-to-end system analysis
por: Raj, Ritik, et al.
Publicado: (2025)

Silicon Showdown: Performance, Efficiency, and Ecosystem Barriers in Consumer-Grade LLM Inference
por: Javat, Abdurrahman, et al.
Publicado: (2026)

CXL-Interference: Analysis and Characterization in Modern Computer Systems
por: Mao, Shunyu, et al.
Publicado: (2024)

Titanus: Enabling KV Cache Pruning and Quantization On-the-Fly for LLM Acceleration
por: Chen, Peilin, et al.
Publicado: (2025)

JSPIM: A Skew-Aware PIM Accelerator for High-Performance Databases Join and Select Operations
por: Tajdari, Sabiha, et al.
Publicado: (2025)

WaveTune: Wave-aware Bilinear Modeling for Efficient GPU Kernel Auto-tuning
por: Zhang, Kaixuan, et al.
Publicado: (2026)

PipeWeave: Synergizing Analytical and Learning Models for Unified GPU Performance Prediction
por: Zhang, Kaixuan, et al.
Publicado: (2026)

VEDA: Efficient LLM Generation Through Voting-based KV Cache Eviction and Dataflow-flexible Accelerator
por: Wang, Zhican, et al.
Publicado: (2025)

ZipServ: Fast and Memory-Efficient LLM Inference with Hardware-Aware Lossless Compression
por: Fan, Ruibo, et al.
Publicado: (2026)

A TRRIP Down Memory Lane: Temperature-Based Re-Reference Interval Prediction For Instruction Caching
por: Kao, Henry, et al.
Publicado: (2025)