Neural Point Cloud Diffusion for Disentangled 3D Shape and Appearance Generation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917737801973760 |
|---|---|
| author | Schröppel, Philipp Wewer, Christopher Lenssen, Jan Eric Ilg, Eddy Brox, Thomas |
| author_facet | Schröppel, Philipp Wewer, Christopher Lenssen, Jan Eric Ilg, Eddy Brox, Thomas |
| contents | Controllable generation of 3D assets is important for many practical applications like content creation in movies, games and engineering, as well as in AR/VR. Recently, diffusion models have shown remarkable results in generation quality of 3D objects. However, none of the existing models enable disentangled generation to control the shape and appearance separately. For the first time, we present a suitable representation for 3D diffusion models to enable such disentanglement by introducing a hybrid point cloud and neural radiance field approach. We model a diffusion process over point positions jointly with a high-dimensional feature space for a local density and radiance decoder. While the point positions represent the coarse shape of the object, the point features allow modeling the geometry and appearance details. This disentanglement enables us to sample both independently and therefore to control both separately. Our approach sets a new state of the art in generation compared to previous disentanglement-capable methods by reduced FID scores of 30-90% and is on-par with other non disentanglement-capable state-of-the art methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_14124 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Neural Point Cloud Diffusion for Disentangled 3D Shape and Appearance Generation Schröppel, Philipp Wewer, Christopher Lenssen, Jan Eric Ilg, Eddy Brox, Thomas Computer Vision and Pattern Recognition Controllable generation of 3D assets is important for many practical applications like content creation in movies, games and engineering, as well as in AR/VR. Recently, diffusion models have shown remarkable results in generation quality of 3D objects. However, none of the existing models enable disentangled generation to control the shape and appearance separately. For the first time, we present a suitable representation for 3D diffusion models to enable such disentanglement by introducing a hybrid point cloud and neural radiance field approach. We model a diffusion process over point positions jointly with a high-dimensional feature space for a local density and radiance decoder. While the point positions represent the coarse shape of the object, the point features allow modeling the geometry and appearance details. This disentanglement enables us to sample both independently and therefore to control both separately. Our approach sets a new state of the art in generation compared to previous disentanglement-capable methods by reduced FID scores of 30-90% and is on-par with other non disentanglement-capable state-of-the art methods. |
| title | Neural Point Cloud Diffusion for Disentangled 3D Shape and Appearance Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2312.14124 |