3D Scene Understanding Through Local Random Access Sequence Modeling
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916675040837632 |
|---|---|
| author | Lee, Wanhee Kotar, Klemen Venkatesh, Rahul Mysore Watrous, Jared Chen, Honglin Aw, Khai Loong Yamins, Daniel L. K. |
| author_facet | Lee, Wanhee Kotar, Klemen Venkatesh, Rahul Mysore Watrous, Jared Chen, Honglin Aw, Khai Loong Yamins, Daniel L. K. |
| contents | 3D scene understanding from single images is a pivotal problem in computer vision with numerous downstream applications in graphics, augmented reality, and robotics. While diffusion-based modeling approaches have shown promise, they often struggle to maintain object and scene consistency, especially in complex real-world scenarios. To address these limitations, we propose an autoregressive generative approach called Local Random Access Sequence (LRAS) modeling, which uses local patch quantization and randomly ordered sequence generation. By utilizing optical flow as an intermediate representation for 3D scene editing, our experiments demonstrate that LRAS achieves state-of-the-art novel view synthesis and 3D object manipulation capabilities. Furthermore, we show that our framework naturally extends to self-supervised depth estimation through a simple modification of the sequence design. By achieving strong performance on multiple 3D scene understanding tasks, LRAS provides a unified and effective framework for building the next generation of 3D vision models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_03875 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | 3D Scene Understanding Through Local Random Access Sequence Modeling Lee, Wanhee Kotar, Klemen Venkatesh, Rahul Mysore Watrous, Jared Chen, Honglin Aw, Khai Loong Yamins, Daniel L. K. Computer Vision and Pattern Recognition 3D scene understanding from single images is a pivotal problem in computer vision with numerous downstream applications in graphics, augmented reality, and robotics. While diffusion-based modeling approaches have shown promise, they often struggle to maintain object and scene consistency, especially in complex real-world scenarios. To address these limitations, we propose an autoregressive generative approach called Local Random Access Sequence (LRAS) modeling, which uses local patch quantization and randomly ordered sequence generation. By utilizing optical flow as an intermediate representation for 3D scene editing, our experiments demonstrate that LRAS achieves state-of-the-art novel view synthesis and 3D object manipulation capabilities. Furthermore, we show that our framework naturally extends to self-supervised depth estimation through a simple modification of the sequence design. By achieving strong performance on multiple 3D scene understanding tasks, LRAS provides a unified and effective framework for building the next generation of 3D vision models. |
| title | 3D Scene Understanding Through Local Random Access Sequence Modeling |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.03875 |