A Quantitative Evaluation of Score Distillation Sampling Based Text-to-3D
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910348314935296 |
|---|---|
| author | Fei, Xiaohan Parameshwara, Chethan Mo, Jiawei Li, Xiaolong Swaminathan, Ashwin Taylor, CJ Favaro, Paolo Soatto, Stefano |
| author_facet | Fei, Xiaohan Parameshwara, Chethan Mo, Jiawei Li, Xiaolong Swaminathan, Ashwin Taylor, CJ Favaro, Paolo Soatto, Stefano |
| contents | The development of generative models that create 3D content from a text prompt has made considerable strides thanks to the use of the score distillation sampling (SDS) method on pre-trained diffusion models for image generation. However, the SDS method is also the source of several artifacts, such as the Janus problem, the misalignment between the text prompt and the generated 3D model, and 3D model inaccuracies. While existing methods heavily rely on the qualitative assessment of these artifacts through visual inspection of a limited set of samples, in this work we propose more objective quantitative evaluation metrics, which we cross-validate via human ratings, and show analysis of the failure cases of the SDS technique. We demonstrate the effectiveness of this analysis by designing a novel computationally efficient baseline model that achieves state-of-the-art performance on the proposed metrics while addressing all the above-mentioned artifacts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_18780 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Quantitative Evaluation of Score Distillation Sampling Based Text-to-3D Fei, Xiaohan Parameshwara, Chethan Mo, Jiawei Li, Xiaolong Swaminathan, Ashwin Taylor, CJ Favaro, Paolo Soatto, Stefano Computer Vision and Pattern Recognition The development of generative models that create 3D content from a text prompt has made considerable strides thanks to the use of the score distillation sampling (SDS) method on pre-trained diffusion models for image generation. However, the SDS method is also the source of several artifacts, such as the Janus problem, the misalignment between the text prompt and the generated 3D model, and 3D model inaccuracies. While existing methods heavily rely on the qualitative assessment of these artifacts through visual inspection of a limited set of samples, in this work we propose more objective quantitative evaluation metrics, which we cross-validate via human ratings, and show analysis of the failure cases of the SDS technique. We demonstrate the effectiveness of this analysis by designing a novel computationally efficient baseline model that achieves state-of-the-art performance on the proposed metrics while addressing all the above-mentioned artifacts. |
| title | A Quantitative Evaluation of Score Distillation Sampling Based Text-to-3D |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2402.18780 |