RayZer: A Self-supervised Large View Synthesis Model
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916716118802432 |
|---|---|
| author | Jiang, Hanwen Tan, Hao Wang, Peng Jin, Haian Zhao, Yue Bi, Sai Zhang, Kai Luan, Fujun Sunkavalli, Kalyan Huang, Qixing Pavlakos, Georgios |
| author_facet | Jiang, Hanwen Tan, Hao Wang, Peng Jin, Haian Zhao, Yue Bi, Sai Zhang, Kai Luan, Fujun Sunkavalli, Kalyan Huang, Qixing Pavlakos, Georgios |
| contents | We present RayZer, a self-supervised multi-view 3D Vision model trained without any 3D supervision, i.e., camera poses and scene geometry, while exhibiting emerging 3D awareness. Concretely, RayZer takes unposed and uncalibrated images as input, recovers camera parameters, reconstructs a scene representation, and synthesizes novel views. During training, RayZer relies solely on its self-predicted camera poses to render target views, eliminating the need for any ground-truth camera annotations and allowing RayZer to be trained with 2D image supervision. The emerging 3D awareness of RayZer is attributed to two key factors. First, we design a self-supervised framework, which achieves 3D-aware auto-encoding of input images by disentangling camera and scene representations. Second, we design a transformer-based model in which the only 3D prior is the ray structure, connecting camera, pixel, and scene simultaneously. RayZer demonstrates comparable or even superior novel view synthesis performance than ``oracle'' methods that rely on pose annotations in both training and testing. Project: https://hwjiang1510.github.io/RayZer/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_00702 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RayZer: A Self-supervised Large View Synthesis Model Jiang, Hanwen Tan, Hao Wang, Peng Jin, Haian Zhao, Yue Bi, Sai Zhang, Kai Luan, Fujun Sunkavalli, Kalyan Huang, Qixing Pavlakos, Georgios Computer Vision and Pattern Recognition We present RayZer, a self-supervised multi-view 3D Vision model trained without any 3D supervision, i.e., camera poses and scene geometry, while exhibiting emerging 3D awareness. Concretely, RayZer takes unposed and uncalibrated images as input, recovers camera parameters, reconstructs a scene representation, and synthesizes novel views. During training, RayZer relies solely on its self-predicted camera poses to render target views, eliminating the need for any ground-truth camera annotations and allowing RayZer to be trained with 2D image supervision. The emerging 3D awareness of RayZer is attributed to two key factors. First, we design a self-supervised framework, which achieves 3D-aware auto-encoding of input images by disentangling camera and scene representations. Second, we design a transformer-based model in which the only 3D prior is the ray structure, connecting camera, pixel, and scene simultaneously. RayZer demonstrates comparable or even superior novel view synthesis performance than ``oracle'' methods that rely on pose annotations in both training and testing. Project: https://hwjiang1510.github.io/RayZer/ |
| title | RayZer: A Self-supervised Large View Synthesis Model |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.00702 |