MeshUp: Multi-Target Mesh Deformation via Blended Score Distillation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929708618219520 |
|---|---|
| author | Kim, Hyunwoo Lang, Itai Aigerman, Noam Groueix, Thibault Kim, Vladimir G. Hanocka, Rana |
| author_facet | Kim, Hyunwoo Lang, Itai Aigerman, Noam Groueix, Thibault Kim, Vladimir G. Hanocka, Rana |
| contents | We propose MeshUp, a technique that deforms a 3D mesh towards multiple target concepts, and intuitively controls the region where each concept is expressed. Conveniently, the concepts can be defined as either text queries, e.g., "a dog" and "a turtle," or inspirational images, and the local regions can be selected as any number of vertices on the mesh. We can effectively control the influence of the concepts and mix them together using a novel score distillation approach, referred to as the Blended Score Distillation (BSD). BSD operates on each attention layer of the denoising U-Net of a diffusion model as it extracts and injects the per-objective activations into a unified denoising pipeline from which the deformation gradients are calculated. To localize the expression of these activations, we create a probabilistic Region of Interest (ROI) map on the surface of the mesh, and turn it into 3D-consistent masks that we use to control the expression of these activations. We demonstrate the effectiveness of BSD empirically and show that it can deform various meshes towards multiple objectives. Our project page is at https://threedle.github.io/MeshUp. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_14899 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MeshUp: Multi-Target Mesh Deformation via Blended Score Distillation Kim, Hyunwoo Lang, Itai Aigerman, Noam Groueix, Thibault Kim, Vladimir G. Hanocka, Rana Computer Vision and Pattern Recognition Graphics We propose MeshUp, a technique that deforms a 3D mesh towards multiple target concepts, and intuitively controls the region where each concept is expressed. Conveniently, the concepts can be defined as either text queries, e.g., "a dog" and "a turtle," or inspirational images, and the local regions can be selected as any number of vertices on the mesh. We can effectively control the influence of the concepts and mix them together using a novel score distillation approach, referred to as the Blended Score Distillation (BSD). BSD operates on each attention layer of the denoising U-Net of a diffusion model as it extracts and injects the per-objective activations into a unified denoising pipeline from which the deformation gradients are calculated. To localize the expression of these activations, we create a probabilistic Region of Interest (ROI) map on the surface of the mesh, and turn it into 3D-consistent masks that we use to control the expression of these activations. We demonstrate the effectiveness of BSD empirically and show that it can deform various meshes towards multiple objectives. Our project page is at https://threedle.github.io/MeshUp. |
| title | MeshUp: Multi-Target Mesh Deformation via Blended Score Distillation |
| topic | Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2408.14899 |