Cover Image

Toward Robust and Efficient ML-Based GPU Caching for Modern Inference

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Peng, Zhang, Jiaji, Zhao, Hailiang, Zhang, Yirong, Chen, Shenyao, Yu, Jiahong, Tang, Xueyan, Wang, Yixuan, Li, Hao, Zou, Jianping, Xiong, Gang, Chow, Kingsum, He, Shuibing, Deng, Shuiguang
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Robustifying Learning-Augmented Caching Efficiently without Compromising 1-Consistency
by: Chen, Peng, et al.
Published: (2025)

Towards Optimal Robustness in Learning-Augmented Paging
by: Chen, Peng, et al.
Published: (2026)

SegQuant: A Semantics-Aware and Generalizable Quantization Framework for Diffusion Models
by: Zhang, Jiaji, et al.
Published: (2025)

CADRef: Robust Out-of-Distribution Detection via Class-Aware Decoupled Relative Feature Leveraging
by: Ling, Zhiwei, et al.
Published: (2025)

Morphis: SLO-Aware Resource Scheduling for Microservices with Time-Varying Call Graphs
by: Tang, Yu, et al.
Published: (2026)

Scene-Aware Latency Estimation for Microservices via Multi-Scale Graph Fusion
by: Sun, Zhichao, et al.
Published: (2026)

Learning-Augmented Algorithms for the Bahncard Problem
by: Zhao, Hailiang, et al.
Published: (2024)

Data-Locality-Aware Task Assignment and Scheduling for Distributed Job Executions
by: Zhao, Hailiang, et al.
Published: (2024)

Reliable Microservice Tail Latency Prediction via Decoupled Dual-Stream Learning and Gradient Modulation
by: Qian, Wenzhuo, et al.
Published: (2025)

TrackTeller: Temporal Multimodal 3D Grounding for Behavior-Dependent Object References
by: Yu, Jiahong, et al.
Published: (2025)

SecCoder: Towards Generalizable and Robust Secure Code Generation
by: Zhang, Boyu, et al.
Published: (2024)

A Heavy-Load-Enhanced and Changeable-Periodicity-Perceived Workload Prediction Network
by: Chen, Feiyi, et al.
Published: (2023)

HopGNN: Boosting Distributed GNN Training Efficiency via Feature-Centric Model Migration
by: Chen, Weijian, et al.
Published: (2024)

GRAB-ANNS: High-Throughput Indexing and Hybrid Search via GPU-Native Bucketing
by: Zhao, Xinkui, et al.
Published: (2026)

Non-markovian neural quantum propagator and its application to the simulation of ultrafast nonlinear spectra
by: Zhang, Jiaji, et al.
Published: (2024)

CLMTracing: Black-box User-level Watermarking for Code Language Model Tracing
by: Zhang, Boyu, et al.
Published: (2025)

Missing-Aware Multimodal Fusion for Unified Microservice Incident Management
by: Qian, Wenzhuo, et al.
Published: (2026)

Adaptive Dual-Weighting Framework for Federated Learning via Out-of-Distribution Detection
by: Ling, Zhiwei, et al.
Published: (2026)

Shiva-DiT: Residual-Based Differentiable Top-$k$ Selection for Efficient Diffusion Transformers
by: Zhang, Jiaji, et al.
Published: (2026)

PeerSync: Accelerating Containerized Service Delivery at the Network Edge
by: Deng, Yinuo, et al.
Published: (2025)

CRouting: Reducing Expensive Distance Calls in Graph-Based Approximate Nearest Neighbor Search
by: Li, Zhenxin, et al.
Published: (2025)

Harvest: Opportunistic Peer-to-Peer GPU Caching for LLM Inference
by: Gopal, Nikhil, et al.
Published: (2026)

Decentralized Proactive Model Offloading and Resource Allocation for Split and Federated Learning
by: Huang, Binbin, et al.
Published: (2024)

Heimdall++: Optimizing GPU Utilization and Pipeline Parallelism for Efficient Single-Pulse Detection
by: Xia, Bingzheng, et al.
Published: (2025)

Frenzy: A Memory-Aware Serverless LLM Training System for Heterogeneous GPU Clusters
by: Chang, Zihan, et al.
Published: (2024)

SlimCaching: Edge Caching of Mixture-of-Experts for Distributed Inference
by: Chen, Qian, et al.
Published: (2025)

GPU-Accelerated Batch-Dynamic Subgraph Matching
by: Qiu, Linshan, et al.
Published: (2024)

Unpacking the ‘Four‐Support’ Model: Towards Contextual Adaptation in HCV Care Cascade
by: Hailiang Xie, et al.
Published: (2025)

MiniKV: Pushing the Limits of LLM Inference via 2-Bit Layer-Discriminative KV Cache
by: Sharma, Akshat, et al.
Published: (2024)

2DIO: A Cache-Accurate Storage Microbenchmark
by: Wang, Yirong, et al.
Published: (2026)

Twin-Space Representation of Classical Mapping Model in the Constraint Phase Space Representation: Numerically Exact Approach to Open Quantum Systems
by: Zhang, Jiaji, et al.
Published: (2025)

Towards Resource-Efficient Serverless LLM Inference with SLINFER
by: Xu, Chuhao, et al.
Published: (2025)

XFMNet: Decoding Cross-Site and Nonstationary Water Patterns via Stepwise Multimodal Fusion for Long-Term Water Quality Forecasting
by: Wang, Ziqi, et al.
Published: (2025)

CodeGlance: Understanding Code Reasoning Challenges in LLMs through Multi-Dimensional Feature Analysis
by: Wang, Yunkun, et al.
Published: (2026)

Synergizing Large Language Models and Task-specific Models for Time Series Anomaly Detection
by: Chen, Feiyi, et al.
Published: (2025)

SuperPose: Improved 6D Pose Estimation with Robust Tracking and Mask-Free Initialization
by: Deng, Yu, et al.
Published: (2024)

CORM: Cache Optimization with Recent Message for Large Language Model Inference
by: Dai, Jincheng, et al.
Published: (2024)

From Prefix Cache to Fusion RAG Cache: Accelerating LLM Inference in Retrieval-Augmented Generation
by: Wang, Jiahao, et al.
Published: (2026)

How institutional pressures on green innovation are perceived by firms? The role of board social ties
by: Hailiang Zou, et al.
Published: (2024)

Predictive Multi-Tier Memory Management for KV Cache in Large-Scale GPU Inference
by: Ganjihal, Sanjeev Rao
Published: (2026)