ArtiFixer: Enhancing and Extending 3D Reconstruction with Auto-Regressive Diffusion Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910193808310272 |
|---|---|
| author | de Lutio, Riccardo Fischer, Tobias Chang, Yen-Yu Zhang, Yuxuan Wu, Jay Zhangjie Ren, Xuanchi Shen, Tianchang Tothova, Katarina Gojcic, Zan Turki, Haithem |
| author_facet | de Lutio, Riccardo Fischer, Tobias Chang, Yen-Yu Zhang, Yuxuan Wu, Jay Zhangjie Ren, Xuanchi Shen, Tianchang Tothova, Katarina Gojcic, Zan Turki, Haithem |
| contents | Per-scene optimization methods such as 3D Gaussian Splatting provide state-of-the-art novel view synthesis quality but extrapolate poorly to under-observed areas. Methods that leverage generative priors to correct artifacts in these areas hold promise but currently suffer from two shortcomings. The first is scalability, as existing methods use image diffusion models or bidirectional video models that are limited in the number of views they can generate in a single pass (and thus require a costly iterative distillation process for consistency). The second is quality itself, as generators used in prior work tend to produce outputs that are inconsistent with existing scene content and fail entirely in completely unobserved regions. To solve these, we propose a two-stage pipeline that leverages two key insights. First, we train a powerful bidirectional generative model with a novel opacity mixing strategy that encourages consistency with existing observations while retaining the model's ability to extrapolate novel content in unseen areas. Second, we distill it into a causal auto-regressive model that generates hundreds of frames in a single pass. This model can directly produce novel views or serve as pseudo-supervision to improve the underlying 3D representation in a simple and highly efficient manner. We evaluate our method extensively and demonstrate that it can generate plausible reconstructions in scenarios where existing approaches fail completely. When measured on commonly benchmarked datasets, we outperform all existing baselines by a wide margin, exceeding prior state-of-the-art methods by 1-3 dB PSNR. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_00492 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | ArtiFixer: Enhancing and Extending 3D Reconstruction with Auto-Regressive Diffusion Models de Lutio, Riccardo Fischer, Tobias Chang, Yen-Yu Zhang, Yuxuan Wu, Jay Zhangjie Ren, Xuanchi Shen, Tianchang Tothova, Katarina Gojcic, Zan Turki, Haithem Computer Vision and Pattern Recognition Artificial Intelligence Graphics Machine Learning Per-scene optimization methods such as 3D Gaussian Splatting provide state-of-the-art novel view synthesis quality but extrapolate poorly to under-observed areas. Methods that leverage generative priors to correct artifacts in these areas hold promise but currently suffer from two shortcomings. The first is scalability, as existing methods use image diffusion models or bidirectional video models that are limited in the number of views they can generate in a single pass (and thus require a costly iterative distillation process for consistency). The second is quality itself, as generators used in prior work tend to produce outputs that are inconsistent with existing scene content and fail entirely in completely unobserved regions. To solve these, we propose a two-stage pipeline that leverages two key insights. First, we train a powerful bidirectional generative model with a novel opacity mixing strategy that encourages consistency with existing observations while retaining the model's ability to extrapolate novel content in unseen areas. Second, we distill it into a causal auto-regressive model that generates hundreds of frames in a single pass. This model can directly produce novel views or serve as pseudo-supervision to improve the underlying 3D representation in a simple and highly efficient manner. We evaluate our method extensively and demonstrate that it can generate plausible reconstructions in scenarios where existing approaches fail completely. When measured on commonly benchmarked datasets, we outperform all existing baselines by a wide margin, exceeding prior state-of-the-art methods by 1-3 dB PSNR. |
| title | ArtiFixer: Enhancing and Extending 3D Reconstruction with Auto-Regressive Diffusion Models |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Graphics Machine Learning |
| url | https://arxiv.org/abs/2603.00492 |