Guardado en:
| Autores principales: | , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2508.11952 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914379094556672 |
|---|---|
| author | Xu, Yueming Zhang, Jiahui Huang, Ze Chen, Yurui Zhou, Yanpeng Chen, Zhenyu Yuan, Yu-Jie Xia, Pengxiang Huang, Guowei Cai, Xinyue Qi, Zhongang Quan, Xingyue Hao, Jianye Xu, Hang Zhang, Li |
| author_facet | Xu, Yueming Zhang, Jiahui Huang, Ze Chen, Yurui Zhou, Yanpeng Chen, Zhenyu Yuan, Yu-Jie Xia, Pengxiang Huang, Guowei Cai, Xinyue Qi, Zhongang Quan, Xingyue Hao, Jianye Xu, Hang Zhang, Li |
| contents | Despite the impressive progress on understanding and generating images shown by the recent unified architectures, the integration of 3D tasks remains challenging and largely unexplored. In this paper, we introduce UniUGG, the first unified understanding and generation framework for 3D modalities. Our unified framework employs an LLM to comprehend and decode sentences and 3D representations. At its core, we propose a spatial decoder leveraging a latent diffusion model to generate high-quality 3D representations. This allows for the generation and imagination of 3D scenes based on a reference image and an arbitrary view transformation, while remaining supports for spatial visual question answering (VQA) tasks. Additionally, we propose a geometric-semantic learning strategy to pretrain the vision encoder. This design jointly captures the input's semantic and geometric cues, enhancing both spatial understanding and generation. Extensive experimental results demonstrate the superiority of our method in visual representation, spatial understanding, and 3D generation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_11952 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | UniUGG: Unified 3D Understanding and Generation via Geometric-Semantic Encoding Xu, Yueming Zhang, Jiahui Huang, Ze Chen, Yurui Zhou, Yanpeng Chen, Zhenyu Yuan, Yu-Jie Xia, Pengxiang Huang, Guowei Cai, Xinyue Qi, Zhongang Quan, Xingyue Hao, Jianye Xu, Hang Zhang, Li Computer Vision and Pattern Recognition Despite the impressive progress on understanding and generating images shown by the recent unified architectures, the integration of 3D tasks remains challenging and largely unexplored. In this paper, we introduce UniUGG, the first unified understanding and generation framework for 3D modalities. Our unified framework employs an LLM to comprehend and decode sentences and 3D representations. At its core, we propose a spatial decoder leveraging a latent diffusion model to generate high-quality 3D representations. This allows for the generation and imagination of 3D scenes based on a reference image and an arbitrary view transformation, while remaining supports for spatial visual question answering (VQA) tasks. Additionally, we propose a geometric-semantic learning strategy to pretrain the vision encoder. This design jointly captures the input's semantic and geometric cues, enhancing both spatial understanding and generation. Extensive experimental results demonstrate the superiority of our method in visual representation, spatial understanding, and 3D generation. |
| title | UniUGG: Unified 3D Understanding and Generation via Geometric-Semantic Encoding |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2508.11952 |