Learning Naturally Aggregated Appearance for Efficient 3D Editing
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929712160309248 |
|---|---|
| author | Cheng, Ka Leong Wang, Qiuyu Shi, Zifan Zheng, Kecheng Xu, Yinghao Ouyang, Hao Chen, Qifeng Shen, Yujun |
| author_facet | Cheng, Ka Leong Wang, Qiuyu Shi, Zifan Zheng, Kecheng Xu, Yinghao Ouyang, Hao Chen, Qifeng Shen, Yujun |
| contents | Neural radiance fields, which represent a 3D scene as a color field and a density field, have demonstrated great progress in novel view synthesis yet are unfavorable for editing due to the implicitness. This work studies the task of efficient 3D editing, where we focus on editing speed and user interactivity. To this end, we propose to learn the color field as an explicit 2D appearance aggregation, also called canonical image, with which users can easily customize their 3D editing via 2D image processing. We complement the canonical image with a projection field that maps 3D points onto 2D pixels for texture query. This field is initialized with a pseudo canonical camera model and optimized with offset regularity to ensure the naturalness of the canonical image. Extensive experiments on different datasets suggest that our representation, dubbed AGAP, well supports various ways of 3D editing (e.g., stylization, instance segmentation, and interactive drawing). Our approach demonstrates remarkable efficiency by being at least 20 times faster per edit compared to existing NeRF-based editing methods. Project page is available at https://felixcheng97.github.io/AGAP/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_06657 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Learning Naturally Aggregated Appearance for Efficient 3D Editing Cheng, Ka Leong Wang, Qiuyu Shi, Zifan Zheng, Kecheng Xu, Yinghao Ouyang, Hao Chen, Qifeng Shen, Yujun Computer Vision and Pattern Recognition Neural radiance fields, which represent a 3D scene as a color field and a density field, have demonstrated great progress in novel view synthesis yet are unfavorable for editing due to the implicitness. This work studies the task of efficient 3D editing, where we focus on editing speed and user interactivity. To this end, we propose to learn the color field as an explicit 2D appearance aggregation, also called canonical image, with which users can easily customize their 3D editing via 2D image processing. We complement the canonical image with a projection field that maps 3D points onto 2D pixels for texture query. This field is initialized with a pseudo canonical camera model and optimized with offset regularity to ensure the naturalness of the canonical image. Extensive experiments on different datasets suggest that our representation, dubbed AGAP, well supports various ways of 3D editing (e.g., stylization, instance segmentation, and interactive drawing). Our approach demonstrates remarkable efficiency by being at least 20 times faster per edit compared to existing NeRF-based editing methods. Project page is available at https://felixcheng97.github.io/AGAP/. |
| title | Learning Naturally Aggregated Appearance for Efficient 3D Editing |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2312.06657 |