BulletGen: Improving 4D Reconstruction with Bullet-Time Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917388106072064 |
|---|---|
| author | Rozumny, Denis Luiten, Jonathon Khan, Numair Schönberger, Johannes Kontschieder, Peter |
| author_facet | Rozumny, Denis Luiten, Jonathon Khan, Numair Schönberger, Johannes Kontschieder, Peter |
| contents | Transforming casually captured, monocular videos into fully immersive dynamic experiences is a highly ill-posed task, and comes with significant challenges, e.g., reconstructing unseen regions, and dealing with the ambiguity in monocular depth estimation. In this work we introduce BulletGen, an approach that takes advantage of generative models to correct errors and complete missing information in a Gaussian-based dynamic scene representation. This is done by aligning the output of a diffusion-based video generation model with the 4D reconstruction at a single frozen "bullet-time" step. The generated frames are then used to supervise the optimization of the 4D Gaussian model. Our method seamlessly blends generative content with both static and dynamic scene components, achieving state-of-the-art results on both novel-view synthesis, and 2D/3D tracking tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_18601 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | BulletGen: Improving 4D Reconstruction with Bullet-Time Generation Rozumny, Denis Luiten, Jonathon Khan, Numair Schönberger, Johannes Kontschieder, Peter Graphics Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning Transforming casually captured, monocular videos into fully immersive dynamic experiences is a highly ill-posed task, and comes with significant challenges, e.g., reconstructing unseen regions, and dealing with the ambiguity in monocular depth estimation. In this work we introduce BulletGen, an approach that takes advantage of generative models to correct errors and complete missing information in a Gaussian-based dynamic scene representation. This is done by aligning the output of a diffusion-based video generation model with the 4D reconstruction at a single frozen "bullet-time" step. The generated frames are then used to supervise the optimization of the 4D Gaussian model. Our method seamlessly blends generative content with both static and dynamic scene components, achieving state-of-the-art results on both novel-view synthesis, and 2D/3D tracking tasks. |
| title | BulletGen: Improving 4D Reconstruction with Bullet-Time Generation |
| topic | Graphics Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2506.18601 |