Pyramid Forcing: Head-Aware Pyramid KV Cache Policy for High-Quality Long Video Generation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910215581990912 |
|---|---|
| author | Chen, Jiayu Tang, Junbei Zhao, Wenbiao Li, Maoliang Luo, Jiayi Zheng, Zihao Yang, Jiawei Luo, Guojie Chen, Xiang |
| author_facet | Chen, Jiayu Tang, Junbei Zhao, Wenbiao Li, Maoliang Luo, Jiayi Zheng, Zihao Yang, Jiawei Luo, Guojie Chen, Xiang |
| contents | Autoregressive video generation enables streaming and open-ended long video synthesis, but still suffers from long-term degradation caused by accumulated errors. Existing KVCache strategies usually apply unified historical-frame retention, implicitly assuming homogeneous historical dependencies across attention heads. We revisit historical-frame attention and reveal three distinct head types: Anchor Heads require broad long-range context, Wave Heads exhibit periodic temporal dependencies, and Veil Heads focus on initial and adjacent frames. Based on this finding, we propose Pyramid Forcing, a head-aware pyramidal KVCache framework that identifies head types offline, assigns behavior-specific cache policies, and supports heterogeneous cache lengths via efficient ragged-cache attention. Experiments on Self Forcing and Causal Forcing show that Pyramid Forcing consistently improves long-horizon generation quality on VBench-Long, increasing the 60-second Self Forcing score from 77.87 to 81.21 while enhancing motion dynamics, visual fidelity, and semantic consistency. Project: https://if-lab-pku.github.io/Pyramid-Forcing/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_13111 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Pyramid Forcing: Head-Aware Pyramid KV Cache Policy for High-Quality Long Video Generation Chen, Jiayu Tang, Junbei Zhao, Wenbiao Li, Maoliang Luo, Jiayi Zheng, Zihao Yang, Jiawei Luo, Guojie Chen, Xiang Computer Vision and Pattern Recognition Autoregressive video generation enables streaming and open-ended long video synthesis, but still suffers from long-term degradation caused by accumulated errors. Existing KVCache strategies usually apply unified historical-frame retention, implicitly assuming homogeneous historical dependencies across attention heads. We revisit historical-frame attention and reveal three distinct head types: Anchor Heads require broad long-range context, Wave Heads exhibit periodic temporal dependencies, and Veil Heads focus on initial and adjacent frames. Based on this finding, we propose Pyramid Forcing, a head-aware pyramidal KVCache framework that identifies head types offline, assigns behavior-specific cache policies, and supports heterogeneous cache lengths via efficient ragged-cache attention. Experiments on Self Forcing and Causal Forcing show that Pyramid Forcing consistently improves long-horizon generation quality on VBench-Long, increasing the 60-second Self Forcing score from 77.87 to 81.21 while enhancing motion dynamics, visual fidelity, and semantic consistency. Project: https://if-lab-pku.github.io/Pyramid-Forcing/. |
| title | Pyramid Forcing: Head-Aware Pyramid KV Cache Policy for High-Quality Long Video Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2605.13111 |