Cover Image

Tutti: Making SSD-Backed KV Cache Practical for Long-Context LLM Serving

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Qiu, Shi, Hu, Yifan, Wang, Xintao, Zhu, Wenhao, Yan, Jianqin, Chen, Hao, Xu, Kaiqiang, Chen, Kai, Zhang, Yiming
Format:	Preprint
Published:	2026
Subjects:	Operating Systems
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Oneiros: KV Cache Optimization through Parameter Remapping for Multi-tenant LLM Serving
by: Li, Ruihao, et al.
Published: (2025)

EVICPRESS: Joint KV-Cache Compression and Eviction for Efficient LLM Serving
by: Feng, Shaoting, et al.
Published: (2025)

ProphetKV: User-Query-Driven Selective Recomputation for Efficient KV Cache Reuse in Retrieval-Augmented Generation
by: Wang, Shihao, et al.
Published: (2026)

AdaptCache: KV Cache Native Storage Hierarchy for Low-Delay and High-Quality Language Model Serving
by: Feng, Shaoting, et al.
Published: (2025)

ContiguousKV: Accelerating LLM Prefill with Granularity-Aligned KV Cache Management
by: Zou, Jing, et al.
Published: (2026)

KV-RM: Regularizing KV-Cache Movement for Static-Graph LLM Serving
by: Zhong, Zhiqing, et al.
Published: (2026)

Towards High-Goodput LLM Serving with Prefill-decode Multiplexing
by: Chen, Yukang, et al.
Published: (2025)

Selective KV-Cache Sharing to Mitigate Timing Side-Channels in LLM Inference
by: Chu, Kexin, et al.
Published: (2025)

Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live
by: Li, Hanchen, et al.
Published: (2025)

PipeANN-Filter: An Efficient Filtered Vector Search System on SSD
by: Guo, Hao, et al.
Published: (2026)

Optimizing SSD Caches for Cloud Block Storage Systems Using Machine Learning Approaches
by: Cheng, Chiyu, et al.
Published: (2024)

DFUSE: Strongly Consistent Write-Back Kernel Caching for Distributed Userspace File Systems
by: Li, Haoyu, et al.
Published: (2025)

Guidelines for Building Indexes on Partially Cache-Coherent CXL Shared Memory
by: Wu, Fangnuo, et al.
Published: (2025)

Breaking the Boundaries of Long-Context LLM Inference: Adaptive KV Management on a Single Commodity GPU
by: Sun, He, et al.
Published: (2025)

XBOF: A Cost-Efficient CXL JBOF with Inter-SSD Compute Resource Sharing
by: Yi, Shushu, et al.
Published: (2025)

Nova: Real-Time Agentic Vision-Language Model Serving with Adaptive Cross-Stage Parallelization
by: Xu, Yuhang, et al.
Published: (2025)

Gromov-Hausdorff Convergence of Spectral Truncations for Quantum Groups
by: Peng, Xintao, et al.
Published: (2026)

FlexServe: A Fast and Secure LLM Serving System for Mobile Devices with Flexible Resource Isolation
by: Wu, Yinpeng, et al.
Published: (2026)

HACache: Leveraging Read Performance with Cache in a Heterogeneous Array
by: Liu, Jialin, et al.
Published: (2026)

Developing Generative AI for Value Co‐Creation: An Intervention‐Based Randomized Field Experiment in a Healthcare Context
by: Yanwu Song, et al.
Published: (2025)

C2CServe: Leveraging NVLink-C2C for Elastic Serverless LLM Serving on MIG
by: Luo, Shutian, et al.
Published: (2026)

Adaptive KV Cache Reuse for Fast Long-Context LLM Serving
by: li, Fei, et al.
Published: (2026)

GoCkpt: Gradient-Assisted Multi-Step overlapped Checkpointing for Efficient LLM Training
by: Zhang, Keyao, et al.
Published: (2025)

Cache is King: Smart Page Eviction with eBPF
by: Zussman, Tal, et al.
Published: (2025)

PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU
by: Song, Yixin, et al.
Published: (2023)

Towards Efficient and Practical GPU Multitasking in the Era of LLM
by: Xing, Jiarong, et al.
Published: (2025)

RTP-LLM: High-Performance Alibaba LLM Inference Engine
by: Tan, Boyu, et al.
Published: (2026)

Idleness is Relative: Exploiting Tool-Call Idle Windows for Offloading in Agentic Systems with MORI
by: Xia, Tian, et al.
Published: (2026)

2DIO: A Cache-Accurate Storage Microbenchmark
by: Wang, Yirong, et al.
Published: (2026)

BLITZSCALE: Fast and Live Large Model Autoscaling with O(1) Host Caching
by: Zhang, Dingyan, et al.
Published: (2024)

Back-and-forth equivalent group von Neumann algebras
by: Goldbring, Isaac, et al.
Published: (2024)

StepCache: Step-Level Reuse with Lightweight Verification and Selective Patching for LLM Serving
by: Nouri, Azam
Published: (2026)

Beyond Money: Incentive Effects of Tokenized Ownership on User Contribution in DAOs
by: Kun Chen, et al.
Published: (2025)

ConZone+: Practical Zoned Flash Storage Emulation for Consumer Devices
by: Yu, Dingcui, et al.
Published: (2025)

DynamicAdaptiveClimb: Adaptive Cache Replacement with Dynamic Resizing
by: Berend, Daniel, et al.
Published: (2025)

Valve: Production Online-Offline Inference Colocation with Jointly-Bounded Preemption Latency and Rate
by: Liu, Fangyue, et al.
Published: (2026)

Flare: Anomaly Diagnostics for Divergent LLM Training in GPU Clusters of Thousand-Plus Scale
by: Cui, Weihao, et al.
Published: (2025)

Idiosyncrasies of Programmable Caching Engines
by: Peixoto, José, et al.
Published: (2026)

vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention
by: Prabhu, Ramya, et al.
Published: (2024)

Optimizing CPU Cache Utilization in Cloud VMs with Accurate Cache Abstraction
by: Tofigh, Mani, et al.
Published: (2025)