PolyOculus: Simultaneous Multi-view Image-based Novel View Synthesis
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911968041893888 |
|---|---|
| author | Yu, Jason J. Aumentado-Armstrong, Tristan Forghani, Fereshteh Derpanis, Konstantinos G. Brubaker, Marcus A. |
| author_facet | Yu, Jason J. Aumentado-Armstrong, Tristan Forghani, Fereshteh Derpanis, Konstantinos G. Brubaker, Marcus A. |
| contents | This paper considers the problem of generative novel view synthesis (GNVS), generating novel, plausible views of a scene given a limited number of known views. Here, we propose a set-based generative model that can simultaneously generate multiple, self-consistent new views, conditioned on any number of views. Our approach is not limited to generating a single image at a time and can condition on a variable number of views. As a result, when generating a large number of views, our method is not restricted to a low-order autoregressive generation approach and is better able to maintain generated image quality over large sets of images. We evaluate our model on standard NVS datasets and show that it outperforms the state-of-the-art image-based GNVS baselines. Further, we show that the model is capable of generating sets of views that have no natural sequential ordering, like loops and binocular trajectories, and significantly outperforms other methods on such tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_17986 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PolyOculus: Simultaneous Multi-view Image-based Novel View Synthesis Yu, Jason J. Aumentado-Armstrong, Tristan Forghani, Fereshteh Derpanis, Konstantinos G. Brubaker, Marcus A. Computer Vision and Pattern Recognition This paper considers the problem of generative novel view synthesis (GNVS), generating novel, plausible views of a scene given a limited number of known views. Here, we propose a set-based generative model that can simultaneously generate multiple, self-consistent new views, conditioned on any number of views. Our approach is not limited to generating a single image at a time and can condition on a variable number of views. As a result, when generating a large number of views, our method is not restricted to a low-order autoregressive generation approach and is better able to maintain generated image quality over large sets of images. We evaluate our model on standard NVS datasets and show that it outperforms the state-of-the-art image-based GNVS baselines. Further, we show that the model is capable of generating sets of views that have no natural sequential ordering, like loops and binocular trajectories, and significantly outperforms other methods on such tasks. |
| title | PolyOculus: Simultaneous Multi-view Image-based Novel View Synthesis |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2402.17986 |