Geometry-Guided Reinforcement Learning for Multi-view Consistent 3D Scene Editing
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908862981865472 |
|---|---|
| author | Wang, Jiyuan Lin, Chunyu Sun, Lei Cao, Zhi Yin, Yuyang Nie, Lang Yuan, Zhenlong Chu, Xiangxiang Wei, Yunchao Liao, Kang Lin, Guosheng |
| author_facet | Wang, Jiyuan Lin, Chunyu Sun, Lei Cao, Zhi Yin, Yuyang Nie, Lang Yuan, Zhenlong Chu, Xiangxiang Wei, Yunchao Liao, Kang Lin, Guosheng |
| contents | Leveraging the priors of 2D diffusion models for 3D editing has emerged as a promising paradigm. However, maintaining multi-view consistency in edited results remains challenging, and the extreme scarcity of 3D-consistent editing paired data renders supervised fine-tuning (SFT), the most effective training strategy for editing tasks, infeasible. In this paper, we observe that, while generating multi-view consistent 3D content is highly challenging, verifying 3D consistency is tractable, naturally positioning reinforcement learning (RL) as a feasible solution. Motivated by this, we propose \textbf{RL3DEdit}, a single-pass framework driven by RL optimization with novel rewards derived from the 3D foundation model, VGGT. Specifically, we leverage VGGT's robust priors learned from massive real-world data, feed the edited images, and utilize the output confidence maps and pose estimation errors as reward signals, effectively anchoring the 2D editing priors onto a 3D-consistent manifold via RL. Extensive experiments demonstrate that RL3DEdit achieves stable multi-view consistency and outperforms state-of-the-art methods in editing quality with high efficiency. To promote the development of 3D editing, we will release the code and model. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_03143 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Geometry-Guided Reinforcement Learning for Multi-view Consistent 3D Scene Editing Wang, Jiyuan Lin, Chunyu Sun, Lei Cao, Zhi Yin, Yuyang Nie, Lang Yuan, Zhenlong Chu, Xiangxiang Wei, Yunchao Liao, Kang Lin, Guosheng Computer Vision and Pattern Recognition Artificial Intelligence Leveraging the priors of 2D diffusion models for 3D editing has emerged as a promising paradigm. However, maintaining multi-view consistency in edited results remains challenging, and the extreme scarcity of 3D-consistent editing paired data renders supervised fine-tuning (SFT), the most effective training strategy for editing tasks, infeasible. In this paper, we observe that, while generating multi-view consistent 3D content is highly challenging, verifying 3D consistency is tractable, naturally positioning reinforcement learning (RL) as a feasible solution. Motivated by this, we propose \textbf{RL3DEdit}, a single-pass framework driven by RL optimization with novel rewards derived from the 3D foundation model, VGGT. Specifically, we leverage VGGT's robust priors learned from massive real-world data, feed the edited images, and utilize the output confidence maps and pose estimation errors as reward signals, effectively anchoring the 2D editing priors onto a 3D-consistent manifold via RL. Extensive experiments demonstrate that RL3DEdit achieves stable multi-view consistency and outperforms state-of-the-art methods in editing quality with high efficiency. To promote the development of 3D editing, we will release the code and model. |
| title | Geometry-Guided Reinforcement Learning for Multi-view Consistent 3D Scene Editing |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2603.03143 |