InstCache: A Predictive Cache for LLM Serving
Fuente:
arXiv
Guardado en:
| Autores principales: | Zou, Longwei, Liu, Yan, Kang, Jiamu, Liu, Tingfeng, Kong, Jiangang, Deng, Yangdong |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
Ejemplares similares
CacheFlow: Efficient LLM Serving with 3D-Parallel KV Cache Restoration
por: Nian, Sean, et al.
Publicado: (2026)
por: Nian, Sean, et al.
Publicado: (2026)
LLM-dCache: Improving Tool-Augmented LLMs with GPT-Driven Localized Data Caching
por: Singh, Simranjit, et al.
Publicado: (2024)
por: Singh, Simranjit, et al.
Publicado: (2024)
PerCache: Predictive Hierarchical Cache for RAG Applications on Mobile Devices
por: Liu, Kaiwei, et al.
Publicado: (2025)
por: Liu, Kaiwei, et al.
Publicado: (2025)
MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool
por: Hu, Cunchen, et al.
Publicado: (2024)
por: Hu, Cunchen, et al.
Publicado: (2024)
MorphServe: Efficient and Workload-Aware LLM Serving via Runtime Quantized Layer Swapping and KV Cache Resizing
por: Su, Zhaoyuan, et al.
Publicado: (2025)
por: Su, Zhaoyuan, et al.
Publicado: (2025)
TinyServe: Query-Aware Cache Selection for Efficient LLM Serving
por: Liu, Dong, et al.
Publicado: (2025)
por: Liu, Dong, et al.
Publicado: (2025)
Adaptive KV Cache Reuse for Fast Long-Context LLM Serving
por: li, Fei, et al.
Publicado: (2026)
por: li, Fei, et al.
Publicado: (2026)
FastCache: Optimizing Multimodal LLM Serving through Lightweight KV-Cache Compression Framework
por: Zhu, Jianian, et al.
Publicado: (2025)
por: Zhu, Jianian, et al.
Publicado: (2025)
BanaServe: Unified KV Cache and Dynamic Module Migration for Balancing Disaggregated LLM Serving in AI Infrastructure
por: He, Yiyuan, et al.
Publicado: (2025)
por: He, Yiyuan, et al.
Publicado: (2025)
DualMap: Enabling Both Cache Affinity and Load Balancing for Distributed LLM Serving
por: Yuan, Ying, et al.
Publicado: (2026)
por: Yuan, Ying, et al.
Publicado: (2026)
TraCT: Disaggregated LLM Serving with CXL Shared Memory KV Cache at Rack-Scale
por: Yoon, Dongha, et al.
Publicado: (2025)
por: Yoon, Dongha, et al.
Publicado: (2025)
TokenDance: Scaling Multi-Agent LLM Serving via Collective KV Cache Sharing
por: Bian, Zhuohang, et al.
Publicado: (2026)
por: Bian, Zhuohang, et al.
Publicado: (2026)
Cache Your Prompt When It's Green: Carbon-Aware Caching for Large Language Model Serving
por: Tian, Yuyang, et al.
Publicado: (2025)
por: Tian, Yuyang, et al.
Publicado: (2025)
DuoServe-MoE: Dual-Phase Expert Prefetch and Caching for LLM Inference QoS Assurance
por: Zhang, Yuning, et al.
Publicado: (2025)
por: Zhang, Yuning, et al.
Publicado: (2025)
Strata: Hierarchical Context Caching for Long Context Language Model Serving
por: Xie, Zhiqiang, et al.
Publicado: (2025)
por: Xie, Zhiqiang, et al.
Publicado: (2025)
KV Cache Compression for Inference Efficiency in LLMs: A Review
por: Liu, Yanyu, et al.
Publicado: (2025)
por: Liu, Yanyu, et al.
Publicado: (2025)
EPIC: Efficient Position-Independent Caching for Serving Large Language Models
por: Hu, Junhao, et al.
Publicado: (2024)
por: Hu, Junhao, et al.
Publicado: (2024)
RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generation
por: Jin, Chao, et al.
Publicado: (2024)
por: Jin, Chao, et al.
Publicado: (2024)
Mell: Memory-Efficient Large Language Model Serving via Multi-GPU KV Cache Management
por: Qianli, Liu, et al.
Publicado: (2025)
por: Qianli, Liu, et al.
Publicado: (2025)
PCR: A Prefetch-Enhanced Cache Reuse System for Low-Latency RAG Serving
por: Wang, Wenfeng, et al.
Publicado: (2026)
por: Wang, Wenfeng, et al.
Publicado: (2026)
FedCache: A Knowledge Cache-driven Federated Learning Architecture for Personalized Edge Intelligence
por: Wu, Zhiyuan, et al.
Publicado: (2023)
por: Wu, Zhiyuan, et al.
Publicado: (2023)
Cost-Efficient LLM Serving in the Cloud: VM Selection with KV Cache Offloading
por: Kim, Kihyun, et al.
Publicado: (2025)
por: Kim, Kihyun, et al.
Publicado: (2025)
Efficient LLM Inference with Activation Checkpointing and Hybrid Caching
por: Lee, Sanghyeon, et al.
Publicado: (2025)
por: Lee, Sanghyeon, et al.
Publicado: (2025)
Adaptive K-PackCache: Cost-Centric Data Caching in Cloud
por: Sarkar, Suvarthi, et al.
Publicado: (2025)
por: Sarkar, Suvarthi, et al.
Publicado: (2025)
Not All Tokens Are Worth Caching: Learning Semantic-Aware Eviction for LLM Prefix Caches
por: Fang, Shaoke, et al.
Publicado: (2026)
por: Fang, Shaoke, et al.
Publicado: (2026)
KV-Runahead: Scalable Causal LLM Inference by Parallel Key-Value Cache Generation
por: Cho, Minsik, et al.
Publicado: (2024)
por: Cho, Minsik, et al.
Publicado: (2024)
ShadowServe: Interference-Free KV Cache Fetching for Distributed Prefix Caching
por: Xiang, Xingyu, et al.
Publicado: (2025)
por: Xiang, Xingyu, et al.
Publicado: (2025)
PolyKV: A Shared Asymmetrically-Compressed KV Cache Pool for Multi-Agent LLM Inference
por: Patel, Ishan, et al.
Publicado: (2026)
por: Patel, Ishan, et al.
Publicado: (2026)
InstGenIE: Generative Image Editing Made Efficient with Mask-aware Caching and Scheduling
por: Jiang, Xiaoxiao, et al.
Publicado: (2025)
por: Jiang, Xiaoxiao, et al.
Publicado: (2025)
ProMoE: Fast MoE-based LLM Serving using Proactive Caching
por: Song, Xiaoniu, et al.
Publicado: (2024)
por: Song, Xiaoniu, et al.
Publicado: (2024)
TokenLake: A Unified Segment-level Prefix Cache Pool for Fine-grained Elastic Long-Context LLM Serving
por: Wu, Bingyang, et al.
Publicado: (2025)
por: Wu, Bingyang, et al.
Publicado: (2025)
Conveyor: Efficient Tool-aware LLM Serving with Tool Partial Execution
por: Xu, Yechen, et al.
Publicado: (2024)
por: Xu, Yechen, et al.
Publicado: (2024)
CacheFL: Privacy-Preserving and Efficient Federated Cache Model Fine-Tuning for Vision-Language Models
por: Yi, Mengjun, et al.
Publicado: (2025)
por: Yi, Mengjun, et al.
Publicado: (2025)
RcLLM: Accelerating Generative Recommendation via Beyond-Prefix KV Caching
por: Zhao, Zhan, et al.
Publicado: (2026)
por: Zhao, Zhan, et al.
Publicado: (2026)
10Cache: Heterogeneous Resource-Aware Tensor Caching and Migration for LLM Training
por: Afroz, Sabiha, et al.
Publicado: (2025)
por: Afroz, Sabiha, et al.
Publicado: (2025)
Predictable LLM Serving on GPU Clusters
por: Darzi, Erfan, et al.
Publicado: (2025)
por: Darzi, Erfan, et al.
Publicado: (2025)
CaPGNN: Optimizing Parallel Graph Neural Network Training with Joint Caching and Resource-Aware Graph Partitioning
por: Song, Xianfeng, et al.
Publicado: (2025)
por: Song, Xianfeng, et al.
Publicado: (2025)
Caching Aided Multi-Tenant Serverless Computing
por: Qiao, Chu, et al.
Publicado: (2024)
por: Qiao, Chu, et al.
Publicado: (2024)
DualScale: Energy-Efficient Disaggregated LLM Serving via Phase-Aware Placement and DVFS
por: Basit, Omar, et al.
Publicado: (2026)
por: Basit, Omar, et al.
Publicado: (2026)
PRESERVE: Prefetching Model Weights and KV-Cache in Distributed LLM Serving
por: Yüzügüler, Ahmet Caner, et al.
Publicado: (2025)
por: Yüzügüler, Ahmet Caner, et al.
Publicado: (2025)
Ejemplares similares
-
CacheFlow: Efficient LLM Serving with 3D-Parallel KV Cache Restoration
por: Nian, Sean, et al.
Publicado: (2026) -
LLM-dCache: Improving Tool-Augmented LLMs with GPT-Driven Localized Data Caching
por: Singh, Simranjit, et al.
Publicado: (2024) -
PerCache: Predictive Hierarchical Cache for RAG Applications on Mobile Devices
por: Liu, Kaiwei, et al.
Publicado: (2025) -
MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool
por: Hu, Cunchen, et al.
Publicado: (2024) -
MorphServe: Efficient and Workload-Aware LLM Serving via Runtime Quantized Layer Swapping and KV Cache Resizing
por: Su, Zhaoyuan, et al.
Publicado: (2025)