Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.20499 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914286454964224 |
|---|---|
| author | Guo, Hang Jia, Zhaoyang Li, Jiahao Li, Bin Cai, Yuanhao Wang, Jiangshan Li, Yawei Lu, Yan |
| author_facet | Guo, Hang Jia, Zhaoyang Li, Jiahao Li, Bin Cai, Yuanhao Wang, Jiangshan Li, Yawei Lu, Yan |
| contents | The autoregressive video diffusion model has recently gained considerable research interest due to its causal modeling and iterative denoising. In this work, we identify that the multi-head self-attention in these models under-utilizes historical frames: approximately 25% heads attend almost exclusively to the current frame, and discarding their KV caches incurs only minor performance degradation. Building upon this, we propose Dummy Forcing, a simple yet effective method to control context accessibility across different heads. Specifically, the proposed heterogeneous memory allocation reduces head-wise context redundancy, accompanied by dynamic head programming to adaptively classify head types. Moreover, we develop a context packing technique to achieve more aggressive cache compression. Without additional training, our Dummy Forcing delivers up to 2.0x speedup over the baseline, supporting video generation at 24.3 FPS with less than 0.5% quality drop. Project page is available at https://csguoh.github.io/project/DummyForcing/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_20499 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Efficient Autoregressive Video Diffusion with Dummy Head Guo, Hang Jia, Zhaoyang Li, Jiahao Li, Bin Cai, Yuanhao Wang, Jiangshan Li, Yawei Lu, Yan Computer Vision and Pattern Recognition The autoregressive video diffusion model has recently gained considerable research interest due to its causal modeling and iterative denoising. In this work, we identify that the multi-head self-attention in these models under-utilizes historical frames: approximately 25% heads attend almost exclusively to the current frame, and discarding their KV caches incurs only minor performance degradation. Building upon this, we propose Dummy Forcing, a simple yet effective method to control context accessibility across different heads. Specifically, the proposed heterogeneous memory allocation reduces head-wise context redundancy, accompanied by dynamic head programming to adaptively classify head types. Moreover, we develop a context packing technique to achieve more aggressive cache compression. Without additional training, our Dummy Forcing delivers up to 2.0x speedup over the baseline, supporting video generation at 24.3 FPS with less than 0.5% quality drop. Project page is available at https://csguoh.github.io/project/DummyForcing/. |
| title | Efficient Autoregressive Video Diffusion with Dummy Head |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2601.20499 |