ChopGrad: Pixel-Wise Losses for Latent Video Diffusion via Truncated Backpropagation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908944986800128 |
|---|---|
| author | Rivkin, Dmitriy Ewen, Parker Gao, Lili Ost, Julian Walz, Stefanie Kangutkar, Rasika Bijelic, Mario Heide, Felix |
| author_facet | Rivkin, Dmitriy Ewen, Parker Gao, Lili Ost, Julian Walz, Stefanie Kangutkar, Rasika Bijelic, Mario Heide, Felix |
| contents | Recent video diffusion models achieve high-quality generation through recurrent frame processing where each frame generation depends on previous frames. However, this recurrent mechanism means that training such models in the pixel domain incurs prohibitive memory costs, as activations accumulate across the entire video sequence. This fundamental limitation also makes fine-tuning these models with pixel-wise losses computationally intractable for long or high-resolution videos. This paper introduces ChopGrad, a truncated backpropagation scheme for video decoding, limiting gradient computation to local frame windows while maintaining global consistency. We provide a theoretical analysis of this approximation and show that it enables efficient fine-tuning with frame-wise losses. ChopGrad reduces training memory from scaling linearly with the number of video frames (full backpropagation) to constant memory, and compares favorably to existing state-of-the-art video diffusion models across a suite of conditional video generation tasks with pixel-wise losses, including video super-resolution, video inpainting, video enhancement of neural-rendered scenes, and controlled driving video generation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_17812 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | ChopGrad: Pixel-Wise Losses for Latent Video Diffusion via Truncated Backpropagation Rivkin, Dmitriy Ewen, Parker Gao, Lili Ost, Julian Walz, Stefanie Kangutkar, Rasika Bijelic, Mario Heide, Felix Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning Recent video diffusion models achieve high-quality generation through recurrent frame processing where each frame generation depends on previous frames. However, this recurrent mechanism means that training such models in the pixel domain incurs prohibitive memory costs, as activations accumulate across the entire video sequence. This fundamental limitation also makes fine-tuning these models with pixel-wise losses computationally intractable for long or high-resolution videos. This paper introduces ChopGrad, a truncated backpropagation scheme for video decoding, limiting gradient computation to local frame windows while maintaining global consistency. We provide a theoretical analysis of this approximation and show that it enables efficient fine-tuning with frame-wise losses. ChopGrad reduces training memory from scaling linearly with the number of video frames (full backpropagation) to constant memory, and compares favorably to existing state-of-the-art video diffusion models across a suite of conditional video generation tasks with pixel-wise losses, including video super-resolution, video inpainting, video enhancement of neural-rendered scenes, and controlled driving video generation. |
| title | ChopGrad: Pixel-Wise Losses for Latent Video Diffusion via Truncated Backpropagation |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2603.17812 |