Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.16132 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917278935678976 |
|---|---|
| author | Cherian, Joel Mathew Bharadwaj, Ashutosh Muralidhara Gupta, Vima Iyer, Anand Padmanabha |
| author_facet | Cherian, Joel Mathew Bharadwaj, Ashutosh Muralidhara Gupta, Vima Iyer, Anand Padmanabha |
| contents | Text-to-video diffusion models deliver impressive results but remain slow because of the sequential denoising of 3D latents. Existing approaches to speed up inference either require expensive model retraining or use heuristic-based step skipping, which struggles to maintain video quality as the number of denoising steps decreases. Our work, CHAI, aims to use cross-inference caching to reduce latency while maintaining video quality. We introduce Cache Attention as an effective method for attending to shared objects/scenes across cross-inference latents. This selective attention mechanism enables effective reuse of cached latents across semantically related prompts, yielding high cache hit rates. We show that it is possible to generate high-quality videos using Cache Attention with as few as 8 denoising steps. When integrated into the overall system, CHAI is 1.65x - 3.35x faster than baseline OpenSora 1.2 while maintaining video quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_16132 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | CHAI: CacHe Attention Inference for text2video Cherian, Joel Mathew Bharadwaj, Ashutosh Muralidhara Gupta, Vima Iyer, Anand Padmanabha Computer Vision and Pattern Recognition Machine Learning Text-to-video diffusion models deliver impressive results but remain slow because of the sequential denoising of 3D latents. Existing approaches to speed up inference either require expensive model retraining or use heuristic-based step skipping, which struggles to maintain video quality as the number of denoising steps decreases. Our work, CHAI, aims to use cross-inference caching to reduce latency while maintaining video quality. We introduce Cache Attention as an effective method for attending to shared objects/scenes across cross-inference latents. This selective attention mechanism enables effective reuse of cached latents across semantically related prompts, yielding high cache hit rates. We show that it is possible to generate high-quality videos using Cache Attention with as few as 8 denoising steps. When integrated into the overall system, CHAI is 1.65x - 3.35x faster than baseline OpenSora 1.2 while maintaining video quality. |
| title | CHAI: CacHe Attention Inference for text2video |
| topic | Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2602.16132 |