Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.21205 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909729944502272 |
|---|---|
| author | Chen, Liuhan Cun, Xiaodong Li, Xiaoyu He, Xianyi Yuan, Shenghai Chen, Jie Shan, Ying Yuan, Li |
| author_facet | Chen, Liuhan Cun, Xiaodong Li, Xiaoyu He, Xianyi Yuan, Shenghai Chen, Jie Shan, Ying Yuan, Li |
| contents | Video inbetweening aims to synthesize intermediate video sequences conditioned on the given start and end frames. Current state-of-the-art methods primarily extend large-scale pre-trained Image-to-Video Diffusion Models (I2V-DMs) by incorporating the end-frame condition via direct fine-tuning or temporally bidirectional sampling. However, the former results in a weak end-frame constraint, while the latter inevitably disrupts the input representation of video frames, leading to suboptimal performance. To improve the end-frame constraint while avoiding disruption of the input representation, we propose a novel video inbetweening framework specific to recent and more powerful transformer-based I2V-DMs, termed EF-VI. It efficiently strengthens the end-frame constraint by utilizing an enhanced injection. This is based on our proposed well-designed lightweight module, termed EF-Net, which encodes only the end frame and expands it into temporally adaptive frame-wise features injected into the I2V-DM. Extensive experiments demonstrate the superiority of our EF-VI compared with other baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_21205 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | EF-VI: Enhancing End-Frame Injection for Video Inbetweening Chen, Liuhan Cun, Xiaodong Li, Xiaoyu He, Xianyi Yuan, Shenghai Chen, Jie Shan, Ying Yuan, Li Computer Vision and Pattern Recognition Video inbetweening aims to synthesize intermediate video sequences conditioned on the given start and end frames. Current state-of-the-art methods primarily extend large-scale pre-trained Image-to-Video Diffusion Models (I2V-DMs) by incorporating the end-frame condition via direct fine-tuning or temporally bidirectional sampling. However, the former results in a weak end-frame constraint, while the latter inevitably disrupts the input representation of video frames, leading to suboptimal performance. To improve the end-frame constraint while avoiding disruption of the input representation, we propose a novel video inbetweening framework specific to recent and more powerful transformer-based I2V-DMs, termed EF-VI. It efficiently strengthens the end-frame constraint by utilizing an enhanced injection. This is based on our proposed well-designed lightweight module, termed EF-Net, which encodes only the end frame and expands it into temporally adaptive frame-wise features injected into the I2V-DM. Extensive experiments demonstrate the superiority of our EF-VI compared with other baselines. |
| title | EF-VI: Enhancing End-Frame Injection for Video Inbetweening |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.21205 |