Adapting Image-to-Video Diffusion Models for Large-Motion Frame Interpolation
Fuente:
arXiv
Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929716818083840 |
|---|---|
| author | Jin, Luoxu Watanabe, Hiroshi |
| author_facet | Jin, Luoxu Watanabe, Hiroshi |
| contents | With the development of video generation models has advanced significantly in recent years, we adopt large-scale image-to-video diffusion models for video frame interpolation. We present a conditional encoder designed to adapt an image-to-video model for large-motion frame interpolation. To enhance performance, we integrate a dual-branch feature extractor and propose a cross-frame attention mechanism that effectively captures both spatial and temporal information, enabling accurate interpolations of intermediate frames. Our approach demonstrates superior performance on the Fréchet Video Distance (FVD) metric when evaluated against other state-of-the-art approaches, particularly in handling large motion scenarios, highlighting advancements in generative-based methodologies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_17042 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Adapting Image-to-Video Diffusion Models for Large-Motion Frame Interpolation Jin, Luoxu Watanabe, Hiroshi Computer Vision and Pattern Recognition With the development of video generation models has advanced significantly in recent years, we adopt large-scale image-to-video diffusion models for video frame interpolation. We present a conditional encoder designed to adapt an image-to-video model for large-motion frame interpolation. To enhance performance, we integrate a dual-branch feature extractor and propose a cross-frame attention mechanism that effectively captures both spatial and temporal information, enabling accurate interpolations of intermediate frames. Our approach demonstrates superior performance on the Fréchet Video Distance (FVD) metric when evaluated against other state-of-the-art approaches, particularly in handling large motion scenarios, highlighting advancements in generative-based methodologies. |
| title | Adapting Image-to-Video Diffusion Models for Large-Motion Frame Interpolation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.17042 |