Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.11512 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916525497122816 |
|---|---|
| author | Zhang, Jiale Jia, Qianxi Liu, Yang Zhang, Wei Wei, Wei Tian, Xin |
| author_facet | Zhang, Jiale Jia, Qianxi Liu, Yang Zhang, Wei Wei, Wei Tian, Xin |
| contents | Stereo video conversion aims to transform monocular videos into immersive stereo format. Despite the advancements in novel view synthesis, it still remains two major challenges: i) difficulty of achieving high-fidelity and stable results, and ii) insufficiency of high-quality stereo video data. In this paper, we introduce SpatialMe, a novel stereo video conversion framework based on depth-warping and blend-inpainting. Specifically, we propose a mask-based hierarchy feature update (MHFU) refiner, which integrate and refine the outputs from designed multi-branch inpainting module, using feature update unit (FUU) and mask mechanism. We also propose a disparity expansion strategy to address the problem of foreground bleeding. Furthermore, we conduct a high-quality real-world stereo video dataset -- StereoV1K, to alleviate the data shortage. It contains 1000 stereo videos captured in real-world at a resolution of 1180 x 1180, covering various indoor and outdoor scenes. Extensive experiments demonstrate the superiority of our approach in generating stereo videos over state-of-the-art methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_11512 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SpatialMe: Stereo Video Conversion Using Depth-Warping and Blend-Inpainting Zhang, Jiale Jia, Qianxi Liu, Yang Zhang, Wei Wei, Wei Tian, Xin Computer Vision and Pattern Recognition Stereo video conversion aims to transform monocular videos into immersive stereo format. Despite the advancements in novel view synthesis, it still remains two major challenges: i) difficulty of achieving high-fidelity and stable results, and ii) insufficiency of high-quality stereo video data. In this paper, we introduce SpatialMe, a novel stereo video conversion framework based on depth-warping and blend-inpainting. Specifically, we propose a mask-based hierarchy feature update (MHFU) refiner, which integrate and refine the outputs from designed multi-branch inpainting module, using feature update unit (FUU) and mask mechanism. We also propose a disparity expansion strategy to address the problem of foreground bleeding. Furthermore, we conduct a high-quality real-world stereo video dataset -- StereoV1K, to alleviate the data shortage. It contains 1000 stereo videos captured in real-world at a resolution of 1180 x 1180, covering various indoor and outdoor scenes. Extensive experiments demonstrate the superiority of our approach in generating stereo videos over state-of-the-art methods. |
| title | SpatialMe: Stereo Video Conversion Using Depth-Warping and Blend-Inpainting |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.11512 |