Framer: Interactive Frame Interpolation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910683269955584 |
|---|---|
| author | Wang, Wen Wang, Qiuyu Zheng, Kecheng Ouyang, Hao Chen, Zhekai Gong, Biao Chen, Hao Shen, Yujun Shen, Chunhua |
| author_facet | Wang, Wen Wang, Qiuyu Zheng, Kecheng Ouyang, Hao Chen, Zhekai Gong, Biao Chen, Hao Shen, Yujun Shen, Chunhua |
| contents | We propose Framer for interactive frame interpolation, which targets producing smoothly transitioning frames between two images as per user creativity. Concretely, besides taking the start and end frames as inputs, our approach supports customizing the transition process by tailoring the trajectory of some selected keypoints. Such a design enjoys two clear benefits. First, incorporating human interaction mitigates the issue arising from numerous possibilities of transforming one image to another, and in turn enables finer control of local motions. Second, as the most basic form of interaction, keypoints help establish the correspondence across frames, enhancing the model to handle challenging cases (e.g., objects on the start and end frames are of different shapes and styles). It is noteworthy that our system also offers an "autopilot" mode, where we introduce a module to estimate the keypoints and refine the trajectory automatically, to simplify the usage in practice. Extensive experimental results demonstrate the appealing performance of Framer on various applications, such as image morphing, time-lapse video generation, cartoon interpolation, etc. The code, the model, and the interface will be released to facilitate further research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_18978 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Framer: Interactive Frame Interpolation Wang, Wen Wang, Qiuyu Zheng, Kecheng Ouyang, Hao Chen, Zhekai Gong, Biao Chen, Hao Shen, Yujun Shen, Chunhua Computer Vision and Pattern Recognition We propose Framer for interactive frame interpolation, which targets producing smoothly transitioning frames between two images as per user creativity. Concretely, besides taking the start and end frames as inputs, our approach supports customizing the transition process by tailoring the trajectory of some selected keypoints. Such a design enjoys two clear benefits. First, incorporating human interaction mitigates the issue arising from numerous possibilities of transforming one image to another, and in turn enables finer control of local motions. Second, as the most basic form of interaction, keypoints help establish the correspondence across frames, enhancing the model to handle challenging cases (e.g., objects on the start and end frames are of different shapes and styles). It is noteworthy that our system also offers an "autopilot" mode, where we introduce a module to estimate the keypoints and refine the trajectory automatically, to simplify the usage in practice. Extensive experimental results demonstrate the appealing performance of Framer on various applications, such as image morphing, time-lapse video generation, cartoon interpolation, etc. The code, the model, and the interface will be released to facilitate further research. |
| title | Framer: Interactive Frame Interpolation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2410.18978 |