InfoFlow KV: Information-Flow-Aware KV Recomputation for Long Context
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915837772824576 |
|---|---|
| author | Teng, Xin Zhang, Canyu Zheng, Shaoyi Zhuo, Danyang Zhou, Tianyi Wang, Shengjie |
| author_facet | Teng, Xin Zhang, Canyu Zheng, Shaoyi Zhuo, Danyang Zhou, Tianyi Wang, Shengjie |
| contents | Retrieval-augmented generation (RAG) for long-context question answering is bottlenecked by inference-time prefilling over large retrieved contexts. A common strategy is to precompute key-value (KV) caches for individual documents and selectively recompute a small subset of tokens to restore global causal dependencies, but existing methods rely on heuristics or representation discrepancies without modeling whether selected tokens can effectively influence generation. We cast selective KV recomputation as an information flow problem and show that a simple attention-norm signal from the query reliably identifies tokens that are both semantically relevant and structurally positioned to propagate information, when computed under an inference-consistent RoPE geometry. We therefore reconstruct global positional assignments for retrieved chunks and introduce an information-flow-guided chunk reordering strategy. Experiments on LLM and VLM benchmarks demonstrate consistent gains over prior methods under comparable efficiency budgets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_05353 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | InfoFlow KV: Information-Flow-Aware KV Recomputation for Long Context Teng, Xin Zhang, Canyu Zheng, Shaoyi Zhuo, Danyang Zhou, Tianyi Wang, Shengjie Machine Learning Retrieval-augmented generation (RAG) for long-context question answering is bottlenecked by inference-time prefilling over large retrieved contexts. A common strategy is to precompute key-value (KV) caches for individual documents and selectively recompute a small subset of tokens to restore global causal dependencies, but existing methods rely on heuristics or representation discrepancies without modeling whether selected tokens can effectively influence generation. We cast selective KV recomputation as an information flow problem and show that a simple attention-norm signal from the query reliably identifies tokens that are both semantically relevant and structurally positioned to propagate information, when computed under an inference-consistent RoPE geometry. We therefore reconstruct global positional assignments for retrieved chunks and introduce an information-flow-guided chunk reordering strategy. Experiments on LLM and VLM benchmarks demonstrate consistent gains over prior methods under comparable efficiency budgets. |
| title | InfoFlow KV: Information-Flow-Aware KV Recomputation for Long Context |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2603.05353 |