Copertina

ARKV: Adaptive and Resource-Efficient KV Cache Management under Limited Memory Budget for Long-Context Inference in LLMs

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Lei, Jianlong, Ilager, Shashikant
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Hardware Architecture Artificial Intelligence Distributed, Parallel, and Cluster Computing Performance
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Adaptive KV Cache Reuse for Fast Long-Context LLM Serving
di: li, Fei, et al.
Pubblicazione: (2026)

Fusing Depthwise and Pointwise Convolutions for Efficient Inference on GPUs
di: Qararyah, Fareed, et al.
Pubblicazione: (2024)

Random Adaptive Cache Placement Policy
di: Ahire, Vrushank, et al.
Pubblicazione: (2025)

Profiling Concurrent Vision Inference Workloads on NVIDIA Jetson -- Extended
di: Chakraborty, Abhinaba, et al.
Pubblicazione: (2025)

ZipServ: Fast and Memory-Efficient LLM Inference with Hardware-Aware Lossless Compression
di: Fan, Ruibo, et al.
Pubblicazione: (2026)

Adaptive Multi-Objective Tiered Storage Configuration for KV Cache in LLM Service
di: Zheng, Xianzhe, et al.
Pubblicazione: (2026)

GigaAPI for GPU Parallelization
di: Suvarna, M., et al.
Pubblicazione: (2025)

Performance Analysis of HPC applications on the Aurora Supercomputer: Exploring the Impact of HBM-Enabled Intel Xeon Max CPUs
di: Ibeid, Huda, et al.
Pubblicazione: (2025)

UPMEM Unleashed: Software Secrets for Speed
di: Chmielewski, Krystian, et al.
Pubblicazione: (2025)

Dissecting the NVIDIA Hopper Architecture through Microbenchmarking and Multiple Level Analysis
di: Luo, Weile, et al.
Pubblicazione: (2025)

Experimental Assessment of Containers Running on Top of Virtual Machines
di: Aqasizade, Hossein, et al.
Pubblicazione: (2024)

Compiler Support for Speculation in Decoupled Access/Execute Architectures
di: Szafarczyk, Robert, et al.
Pubblicazione: (2025)

Exploring and Evaluating Real-world CXL: Use Cases and System Adoption
di: Wang, Xi, et al.
Pubblicazione: (2024)

Parallelizing a modern GPU simulator
di: Huerta, Rodrigo, et al.
Pubblicazione: (2025)

Can Asymmetric Tile Buffering Be Beneficial?
di: Wang, Chengyue, et al.
Pubblicazione: (2025)

Exploiting long vectors with a CFD code: a co-design show case
di: Blancafort, Marc, et al.
Pubblicazione: (2024)

Simopt -- Simulation pass for Speculative Optimisation of FPGA-CAD flow
di: Wadhwa, Eashan, et al.
Pubblicazione: (2024)

PyGim: An Efficient Graph Neural Network Library for Real Processing-In-Memory Architectures
di: Giannoula, Christina, et al.
Pubblicazione: (2024)

Understanding Bottlenecks for Efficiently Serving LLM Inference With KV Offloading
di: Meng, William, et al.
Pubblicazione: (2025)

PiKV: KV Cache Management System for Mixture of Experts
di: Liu, Dong, et al.
Pubblicazione: (2025)

Generic and ML Workloads in an HPC Datacenter: Node Energy, Job Failures, and Node-Job Analysis
di: Chu, Xiaoyu, et al.
Pubblicazione: (2024)

The Hitchhiker's Guide to Programming and Optimizing Cache Coherent Heterogeneous Systems: CXL, NVLink-C2C, and AMD Infinity Fabric
di: Wang, Zixuan, et al.
Pubblicazione: (2024)

Evaluating the Potential of In-Memory Processing to Accelerate Homomorphic Encryption
di: Mwaisela, Mpoki, et al.
Pubblicazione: (2024)

PAM: Processing Across Memory Hierarchy for Efficient KV-centric LLM Serving System
di: Liu, Lian, et al.
Pubblicazione: (2026)

Characterizing and Optimizing LLM Inference Workloads on CPU-GPU Coupled Architectures
di: Vellaisamy, Prabhu, et al.
Pubblicazione: (2025)

DCC: Data-Centric Compilation of Machine Learning Kernels for Processing-In-Memory Architectures
di: Yang, Peiming, et al.
Pubblicazione: (2025)

Cloud to Edge: Benchmarking LLM Inference On Hardware-Accelerated Single-Board Computers
di: Renney, Harri, et al.
Pubblicazione: (2026)

ARCAS: Adaptive Runtime System for Chiplet-Aware Scheduling
di: Fogli, Alessandro, et al.
Pubblicazione: (2025)

Predictive Multi-Tier Memory Management for KV Cache in Large-Scale GPU Inference
di: Ganjihal, Sanjeev Rao
Pubblicazione: (2026)

Vector-Centric Machine Learning Systems: A Cross-Stack Approach
di: Jiang, Wenqi
Pubblicazione: (2025)

Chameleon: Adaptive Caching and Scheduling for Many-Adapter LLM Inference Environments
di: Iliakopoulou, Nikoleta, et al.
Pubblicazione: (2024)

Efficient Hardware Accelerator Based on Medium Granularity Dataflow for SpTRSV
di: Chen, Qian, et al.
Pubblicazione: (2024)

Spira: Exploiting Voxel Data Structural Properties for Efficient Sparse Convolution in Point Cloud Networks
di: Adamopoulos, Dionysios, et al.
Pubblicazione: (2025)

Linear Layouts: Robust Code Generation of Efficient Tensor Computation Using $\mathbb{F}_2$
di: Zhou, Keren, et al.
Pubblicazione: (2025)

FengHuang: Next-Generation Memory Orchestration for AI Inferencing
di: Li, Jiamin, et al.
Pubblicazione: (2025)

Tempus: A Temporally Scalable Resource-Invariant GEMM Streaming Framework for Versal AI Edge
di: Grailoo, M., et al.
Pubblicazione: (2026)

Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCache
di: Lin, Bin, et al.
Pubblicazione: (2024)

TAPA: A Scalable Task-Parallel Dataflow Programming Framework for Modern FPGAs with Co-Optimization of HLS and Physical Design
di: Guo, Licheng, et al.
Pubblicazione: (2022)

Optimizing Attention on GPUs by Exploiting GPU Architectural NUMA Effects
di: Choudhary, Mansi, et al.
Pubblicazione: (2025)

Revisiting Disaggregated Large Language Model Serving for Performance and Energy Implications
di: Li, Jiaxi, et al.
Pubblicazione: (2025)