Chirpy3D: Part-Aware Multi-View Diffusion for Creative Fine-Grained Object Generation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918527055691776 |
|---|---|
| author | Ng, Kam Woh Yang, Jing Sii, Jia Wei Chan, Chee Seng Deng, Jiankang Song, Yi-Zhe Xiang, Tao Zhu, Xiatian |
| author_facet | Ng, Kam Woh Yang, Jing Sii, Jia Wei Chan, Chee Seng Deng, Jiankang Song, Yi-Zhe Xiang, Tao Zhu, Xiatian |
| contents | Understanding and generating the fine-grained structure of objects -- such as birds with species-specific beaks, wings, and tails -- is a long-standing challenge in computer vision. We propose Chirpy3D, a part-aware multi-view diffusion framework that learns a hierarchical part latent space from unposed 2D images, using only off-the-shelf 2D part segmentation masks as spatial guidance -- without requiring any 3D data, camera poses, or manual part annotations. This latent space enables intuitive part-level swapping, interpolation, and zero-shot composition. A self-supervised feature consistency loss further encourages structural alignment across views, allowing coherent generation even with hybrid or unseen part combinations. Our core contribution is the controllable part-aware latent space and multi-view diffusion model. Downstream 3D generation is supported via any differentiable renderer such as NeRF but is orthogonal to the main framework, making Chirpy3D a flexible foundation for creative object generation in the absence of structured 3D data. Code is released at https://github.com/kamwoh/chirpy3d. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_04144 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Chirpy3D: Part-Aware Multi-View Diffusion for Creative Fine-Grained Object Generation Ng, Kam Woh Yang, Jing Sii, Jia Wei Chan, Chee Seng Deng, Jiankang Song, Yi-Zhe Xiang, Tao Zhu, Xiatian Computer Vision and Pattern Recognition Graphics Understanding and generating the fine-grained structure of objects -- such as birds with species-specific beaks, wings, and tails -- is a long-standing challenge in computer vision. We propose Chirpy3D, a part-aware multi-view diffusion framework that learns a hierarchical part latent space from unposed 2D images, using only off-the-shelf 2D part segmentation masks as spatial guidance -- without requiring any 3D data, camera poses, or manual part annotations. This latent space enables intuitive part-level swapping, interpolation, and zero-shot composition. A self-supervised feature consistency loss further encourages structural alignment across views, allowing coherent generation even with hybrid or unseen part combinations. Our core contribution is the controllable part-aware latent space and multi-view diffusion model. Downstream 3D generation is supported via any differentiable renderer such as NeRF but is orthogonal to the main framework, making Chirpy3D a flexible foundation for creative object generation in the absence of structured 3D data. Code is released at https://github.com/kamwoh/chirpy3d. |
| title | Chirpy3D: Part-Aware Multi-View Diffusion for Creative Fine-Grained Object Generation |
| topic | Computer Vision and Pattern Recognition Graphics |
| url | https://arxiv.org/abs/2501.04144 |