Instant3dit: Multiview Inpainting for Fast Editing of 3D Objects
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929609502621696 |
|---|---|
| author | Barda, Amir Gadelha, Matheus Kim, Vladimir G. Aigerman, Noam Bermano, Amit H. Groueix, Thibault |
| author_facet | Barda, Amir Gadelha, Matheus Kim, Vladimir G. Aigerman, Noam Bermano, Amit H. Groueix, Thibault |
| contents | We propose a generative technique to edit 3D shapes, represented as meshes, NeRFs, or Gaussian Splats, in approximately 3 seconds, without the need for running an SDS type of optimization. Our key insight is to cast 3D editing as a multiview image inpainting problem, as this representation is generic and can be mapped back to any 3D representation using the bank of available Large Reconstruction Models. We explore different fine-tuning strategies to obtain both multiview generation and inpainting capabilities within the same diffusion model. In particular, the design of the inpainting mask is an important factor of training an inpainting model, and we propose several masking strategies to mimic the types of edits a user would perform on a 3D shape. Our approach takes 3D generative editing from hours to seconds and produces higher-quality results compared to previous works. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_00518 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Instant3dit: Multiview Inpainting for Fast Editing of 3D Objects Barda, Amir Gadelha, Matheus Kim, Vladimir G. Aigerman, Noam Bermano, Amit H. Groueix, Thibault Computer Vision and Pattern Recognition Graphics We propose a generative technique to edit 3D shapes, represented as meshes, NeRFs, or Gaussian Splats, in approximately 3 seconds, without the need for running an SDS type of optimization. Our key insight is to cast 3D editing as a multiview image inpainting problem, as this representation is generic and can be mapped back to any 3D representation using the bank of available Large Reconstruction Models. We explore different fine-tuning strategies to obtain both multiview generation and inpainting capabilities within the same diffusion model. In particular, the design of the inpainting mask is an important factor of training an inpainting model, and we propose several masking strategies to mimic the types of edits a user would perform on a 3D shape. Our approach takes 3D generative editing from hours to seconds and produces higher-quality results compared to previous works. |
| title | Instant3dit: Multiview Inpainting for Fast Editing of 3D Objects |
| topic | Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2412.00518 |