FlowR: Flowing from Sparse to Dense 3D Reconstructions
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918113942962176 |
|---|---|
| author | Fischer, Tobias Bulò, Samuel Rota Yang, Yung-Hsu Keetha, Nikhil Porzi, Lorenzo Müller, Norman Schwarz, Katja Luiten, Jonathon Pollefeys, Marc Kontschieder, Peter |
| author_facet | Fischer, Tobias Bulò, Samuel Rota Yang, Yung-Hsu Keetha, Nikhil Porzi, Lorenzo Müller, Norman Schwarz, Katja Luiten, Jonathon Pollefeys, Marc Kontschieder, Peter |
| contents | 3D Gaussian splatting enables high-quality novel view synthesis (NVS) at real-time frame rates. However, its quality drops sharply as we depart from the training views. Thus, dense captures are needed to match the high-quality expectations of applications like Virtual Reality (VR). However, such dense captures are very laborious and expensive to obtain. Existing works have explored using 2D generative models to alleviate this requirement by distillation or generating additional training views. These models typically rely on a noise-to-data generative process conditioned only on a handful of reference input views, leading to hallucinations, inconsistent generation results, and subsequent reconstruction artifacts. Instead, we propose a multi-view, flow matching model that learns a flow to directly connect novel view renderings from possibly sparse reconstructions to renderings that we expect from dense reconstructions. This enables augmenting scene captures with consistent, generated views to improve reconstruction quality. Our model is trained on a novel dataset of 3.6M image pairs and can process up to 45 views at 540x960 resolution (91K tokens) on one H100 GPU in a single forward pass. Our pipeline consistently improves NVS in sparse- and dense-view scenarios, leading to higher-quality reconstructions than prior works across multiple, widely-used NVS benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_01647 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FlowR: Flowing from Sparse to Dense 3D Reconstructions Fischer, Tobias Bulò, Samuel Rota Yang, Yung-Hsu Keetha, Nikhil Porzi, Lorenzo Müller, Norman Schwarz, Katja Luiten, Jonathon Pollefeys, Marc Kontschieder, Peter Computer Vision and Pattern Recognition 3D Gaussian splatting enables high-quality novel view synthesis (NVS) at real-time frame rates. However, its quality drops sharply as we depart from the training views. Thus, dense captures are needed to match the high-quality expectations of applications like Virtual Reality (VR). However, such dense captures are very laborious and expensive to obtain. Existing works have explored using 2D generative models to alleviate this requirement by distillation or generating additional training views. These models typically rely on a noise-to-data generative process conditioned only on a handful of reference input views, leading to hallucinations, inconsistent generation results, and subsequent reconstruction artifacts. Instead, we propose a multi-view, flow matching model that learns a flow to directly connect novel view renderings from possibly sparse reconstructions to renderings that we expect from dense reconstructions. This enables augmenting scene captures with consistent, generated views to improve reconstruction quality. Our model is trained on a novel dataset of 3.6M image pairs and can process up to 45 views at 540x960 resolution (91K tokens) on one H100 GPU in a single forward pass. Our pipeline consistently improves NVS in sparse- and dense-view scenarios, leading to higher-quality reconstructions than prior works across multiple, widely-used NVS benchmarks. |
| title | FlowR: Flowing from Sparse to Dense 3D Reconstructions |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.01647 |