ScoutAttention: Efficient KV Cache Offloading via Layer-Ahead CPU Pre-computation for LLM Inference

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Qiuyang, Zhou, Kai, Tang, Ding, Lu, Kai, Li, Cheng, Yang, Zhenyu, Xu, Peng, Wan, Jiguang
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Internet

https://arxiv.org/abs/2603.27138