Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.18255 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909926468616192 |
|---|---|
| author | Azar, Sina Mokhtarzadeh Bahrami, Emad Pallotta, Enrico Francesca, Gianpiero Timofte, Radu Gall, Juergen |
| author_facet | Azar, Sina Mokhtarzadeh Bahrami, Emad Pallotta, Enrico Francesca, Gianpiero Timofte, Radu Gall, Juergen |
| contents | In this work, we investigate diffusion-based video prediction models, which forecast future video frames, for continuous video streams. In this context, the models observe continuously new training samples, and we aim to leverage this to improve their predictions. We thus propose an approach that continuously adapts a pre-trained diffusion model to a video stream. Since fine-tuning the parameters of a large diffusion model is too expensive, we refine the diffusion noise during inference while keeping the model parameters frozen, allowing the model to adaptively determine suitable sampling noise. We term the approach Sequence Adaptive Video Prediction with Diffusion Noise Optimization (SAVi-DNO). To validate our approach, we introduce a new evaluation setting on the Ego4D dataset, focusing on simultaneous adaptation and evaluation on long continuous videos. Empirical results demonstrate improved performance based on FVD, SSIM, and PSNR metrics on long videos of Ego4D and OpenDV-YouTube, as well as videos of UCF-101 and SkyTimelapse, showcasing SAVi-DNO's effectiveness. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_18255 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Sequence-Adaptive Video Prediction in Continuous Streams using Diffusion Noise Optimization Azar, Sina Mokhtarzadeh Bahrami, Emad Pallotta, Enrico Francesca, Gianpiero Timofte, Radu Gall, Juergen Computer Vision and Pattern Recognition In this work, we investigate diffusion-based video prediction models, which forecast future video frames, for continuous video streams. In this context, the models observe continuously new training samples, and we aim to leverage this to improve their predictions. We thus propose an approach that continuously adapts a pre-trained diffusion model to a video stream. Since fine-tuning the parameters of a large diffusion model is too expensive, we refine the diffusion noise during inference while keeping the model parameters frozen, allowing the model to adaptively determine suitable sampling noise. We term the approach Sequence Adaptive Video Prediction with Diffusion Noise Optimization (SAVi-DNO). To validate our approach, we introduce a new evaluation setting on the Ego4D dataset, focusing on simultaneous adaptation and evaluation on long continuous videos. Empirical results demonstrate improved performance based on FVD, SSIM, and PSNR metrics on long videos of Ego4D and OpenDV-YouTube, as well as videos of UCF-101 and SkyTimelapse, showcasing SAVi-DNO's effectiveness. |
| title | Sequence-Adaptive Video Prediction in Continuous Streams using Diffusion Noise Optimization |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2511.18255 |