Lay-A-Scene: Personalized 3D Object Arrangement Using Text-to-Image Priors
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913376863518720 |
|---|---|
| author | Rahamim, Ohad Segev, Hilit Achituve, Idan Atzmon, Yuval Kasten, Yoni Chechik, Gal |
| author_facet | Rahamim, Ohad Segev, Hilit Achituve, Idan Atzmon, Yuval Kasten, Yoni Chechik, Gal |
| contents | Generating 3D visual scenes is at the forefront of visual generative AI, but current 3D generation techniques struggle with generating scenes with multiple high-resolution objects. Here we introduce Lay-A-Scene, which solves the task of Open-set 3D Object Arrangement, effectively arranging unseen objects. Given a set of 3D objects, the task is to find a plausible arrangement of these objects in a scene. We address this task by leveraging pre-trained text-to-image models. We personalize the model and explain how to generate images of a scene that contains multiple predefined objects without neglecting any of them. Then, we describe how to infer the 3D poses and arrangement of objects from a 2D generated image by finding a consistent projection of objects onto the 2D scene. We evaluate the quality of Lay-A-Scene using 3D objects from Objaverse and human raters and find that it often generates coherent and feasible 3D object arrangements. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_00687 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Lay-A-Scene: Personalized 3D Object Arrangement Using Text-to-Image Priors Rahamim, Ohad Segev, Hilit Achituve, Idan Atzmon, Yuval Kasten, Yoni Chechik, Gal Computer Vision and Pattern Recognition Generating 3D visual scenes is at the forefront of visual generative AI, but current 3D generation techniques struggle with generating scenes with multiple high-resolution objects. Here we introduce Lay-A-Scene, which solves the task of Open-set 3D Object Arrangement, effectively arranging unseen objects. Given a set of 3D objects, the task is to find a plausible arrangement of these objects in a scene. We address this task by leveraging pre-trained text-to-image models. We personalize the model and explain how to generate images of a scene that contains multiple predefined objects without neglecting any of them. Then, we describe how to infer the 3D poses and arrangement of objects from a 2D generated image by finding a consistent projection of objects onto the 2D scene. We evaluate the quality of Lay-A-Scene using 3D objects from Objaverse and human raters and find that it often generates coherent and feasible 3D object arrangements. |
| title | Lay-A-Scene: Personalized 3D Object Arrangement Using Text-to-Image Priors |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2406.00687 |