Image de couverture de livre

KVSwap: Disk-aware KV Cache Offloading for Long-Context On-device Inference

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhang, Huawei, Xia, Chunwei, Wang, Zheng
Format:	Preprint
Publié:	2025
Sujets:	Distributed, Parallel, and Cluster Computing Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

DUAL-BLADE: Dual-Path NVMe-Direct KV-Cache Offloading for Edge LLM Inference
par: Jeong, Bodon, et autres
Publié: (2026)

FairKV: Balancing Per-Head KV Cache for Fast Multi-GPU Inference
par: Zhao, Bingzhe, et autres
Publié: (2025)

Leyline: KV Cache Directives for Agentic Inference
par: Ma, Bole, et autres
Publié: (2026)

FlowKV: A Disaggregated Inference Framework with Low-Latency KV Cache Transfer and Load-Aware Scheduling
par: Li, Weiqing, et autres
Publié: (2025)

PackKV: Reducing KV Cache Memory Footprint through LLM-Aware Lossy Compression
par: Jiang, Bo, et autres
Publié: (2025)

ARKV: Adaptive and Resource-Efficient KV Cache Management under Limited Memory Budget for Long-Context Inference in LLMs
par: Lei, Jianlong, et autres
Publié: (2026)

KVComp: A High-Performance, LLM-Aware, Lossy Compression Framework for KV Cache
par: Jiang, Bo, et autres
Publié: (2025)

A Survey on Large Language Model Acceleration based on KV Cache Management
par: Li, Haoyang, et autres
Publié: (2024)

KV-Runahead: Scalable Causal LLM Inference by Parallel Key-Value Cache Generation
par: Cho, Minsik, et autres
Publié: (2024)

PiKV: KV Cache Management System for Mixture of Experts
par: Liu, Dong, et autres
Publié: (2025)

DynaKV: Enabling Accurate and Efficient Long-Sequence LLM Decoding on Smartphones
par: Wang, Tuowei, et autres
Publié: (2025)

ShadowServe: Interference-Free KV Cache Fetching for Distributed Prefix Caching
par: Xiang, Xingyu, et autres
Publié: (2025)

KV Cache Compression for Inference Efficiency in LLMs: A Review
par: Liu, Yanyu, et autres
Publié: (2025)

SparOA: Sparse and Operator-aware Hybrid Scheduling for Edge DNN Inference
par: Zhang, Ziyang, et autres
Publié: (2025)

KVCache Cache in the Wild: Characterizing and Optimizing KVCache Cache at a Large Cloud Provider
par: Wang, Jiahao, et autres
Publié: (2025)

PRESERVE: Prefetching Model Weights and KV-Cache in Distributed LLM Serving
par: Yüzügüler, Ahmet Caner, et autres
Publié: (2025)

MoSKA: Mixture of Shared KV Attention for Efficient Long-Sequence LLM Inference
par: Rhee, Myunghyun, et autres
Publié: (2025)

A Model Aware AIGC Task Offloading Algorithm in IIoT Edge Computing
par: Wang, Xin, et autres
Publié: (2025)

Adaptive KV Cache Reuse for Fast Long-Context LLM Serving
par: li, Fei, et autres
Publié: (2026)

Joint Resource Optimization, Computation Offloading and Resource Slicing for Multi-Edge Traffic-Cognitive Networks
par: Xiaoyang, Ting, et autres
Publié: (2024)

PIPO: Pipelined Offloading for Efficient Inference on Consumer Devices
par: Liu, Yangyijian, et autres
Publié: (2025)

Inference Offloading for Cost-Sensitive Binary Classification at the Edge
par: Moothedath, Vishnu Narayanan, et autres
Publié: (2025)

TCM-Serve: Modality-aware Scheduling for Multimodal Large Language Model Inference
par: Papaioannou, Konstantinos, et autres
Publié: (2026)

Understanding Bottlenecks for Efficiently Serving LLM Inference With KV Offloading
par: Meng, William, et autres
Publié: (2025)

Towards Multi-Model LLM Schedulers: Empirical Insights into Offloading and Preemption
par: Yildiz, Mert, et autres
Publié: (2026)

KAIROS: Stateful, Context-Aware Power-Efficient Agentic Inference Serving
par: Yuan, Yichao, et autres
Publié: (2026)

InfiniPipe: Elastic Pipeline Parallelism for Efficient Variable-Length Long-Context LLM Training
par: Wang, Shiju, et autres
Publié: (2025)

Cost-Efficient LLM Serving in the Cloud: VM Selection with KV Cache Offloading
par: Kim, Kihyun, et autres
Publié: (2025)

KVServe: Service-Aware KV Cache Compression for Communication-Efficient Disaggregated LLM Serving
par: Liu, Zedong, et autres
Publié: (2026)

MatKV: Trading Compute for Flash Storage in LLM Inference
par: Shin, Kun-Woo, et autres
Publié: (2025)

Efficient Multi-Adapter LLM Serving via Cross-Model KV-Cache Reuse with Activated LoRA
par: Li, Allison, et autres
Publié: (2025)

NEO: Saving GPU Memory Crisis with CPU Offloading for Online LLM Inference
par: Jiang, Xuanlin, et autres
Publié: (2024)

InstGenIE: Generative Image Editing Made Efficient with Mask-aware Caching and Scheduling
par: Jiang, Xiaoxiao, et autres
Publié: (2025)

Kascade: A Practical Sparse Attention Method for Long-Context LLM Inference
par: Deshmukh, Dhruv, et autres
Publié: (2025)

Towards Efficient Key-Value Cache Management for Prefix Prefilling in LLM Inference
par: Zhu, Yue, et autres
Publié: (2025)

TinyServe: Query-Aware Cache Selection for Efficient LLM Serving
par: Liu, Dong, et autres
Publié: (2025)

OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization
par: Zhou, Zhongzhu, et autres
Publié: (2026)

SPECTRE: Hybrid Ordinary-Parallel Speculative Serving for Resource-Efficient LLM Inference
par: Xie, Jincheng, et autres
Publié: (2026)

To Offload or Not To Offload: Model-driven Comparison of Edge-native and On-device Processing In the Era of Accelerators
par: Ng, Nathan, et autres
Publié: (2025)

Topology-aware Preemptive Scheduling for Co-located LLM Workloads
par: Zhang, Ping, et autres
Publié: (2024)