JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911699580223488 |
|---|---|
| author | Song, Lin Li, Wenbo Ma, Guoqing Tang, Wei Wang, Bo Zhang, Yuan Yang, Yijun Xiao, Yicheng Liu, Jianhui Zhang, Yanbing Zhang, Guohui Zhang, Wenhu Xu, Hang Jiang, Nan Han, Xin Sun, Haoze Zhang, Maoquan Huang, Haoyang Duan, Nan |
| author_facet | Song, Lin Li, Wenbo Ma, Guoqing Tang, Wei Wang, Bo Zhang, Yuan Yang, Yijun Xiao, Yicheng Liu, Jianhui Zhang, Yanbing Zhang, Guohui Zhang, Wenhu Xu, Hang Jiang, Nan Han, Xin Sun, Haoze Zhang, Maoquan Huang, Haoyang Duan, Nan |
| contents | We present JoyAI-Image, a unified multimodal foundation model for visual understanding, text-to-image generation, and instruction-guided image editing. JoyAI-Image couples a spatially enhanced Multimodal Large Language Model (MLLM) with a Multimodal Diffusion Transformer (MMDiT), allowing perception and generation to interact through a shared multimodal interface. Around this architecture, we build a scalable training recipe that combines unified instruction tuning, long-text rendering supervision, spatially grounded data, and both general and spatial editing signals. This design gives the model broad multimodal capability while strengthening geometry-aware reasoning and controllable visual synthesis. Experiments across understanding, generation, long-text rendering, and editing benchmarks show that JoyAI-Image achieves state-of-the-art or highly competitive performance. More importantly, the bidirectional loop between enhanced understanding, controllable spatial editing, and novel-view-assisted reasoning enables the model to move beyond general visual competence toward stronger spatial intelligence. These results suggest a promising path for unified visual models in downstream applications such as vision-language-action systems and world models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_04128 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generation Song, Lin Li, Wenbo Ma, Guoqing Tang, Wei Wang, Bo Zhang, Yuan Yang, Yijun Xiao, Yicheng Liu, Jianhui Zhang, Yanbing Zhang, Guohui Zhang, Wenhu Xu, Hang Jiang, Nan Han, Xin Sun, Haoze Zhang, Maoquan Huang, Haoyang Duan, Nan Graphics Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Machine Learning We present JoyAI-Image, a unified multimodal foundation model for visual understanding, text-to-image generation, and instruction-guided image editing. JoyAI-Image couples a spatially enhanced Multimodal Large Language Model (MLLM) with a Multimodal Diffusion Transformer (MMDiT), allowing perception and generation to interact through a shared multimodal interface. Around this architecture, we build a scalable training recipe that combines unified instruction tuning, long-text rendering supervision, spatially grounded data, and both general and spatial editing signals. This design gives the model broad multimodal capability while strengthening geometry-aware reasoning and controllable visual synthesis. Experiments across understanding, generation, long-text rendering, and editing benchmarks show that JoyAI-Image achieves state-of-the-art or highly competitive performance. More importantly, the bidirectional loop between enhanced understanding, controllable spatial editing, and novel-view-assisted reasoning enables the model to move beyond general visual competence toward stronger spatial intelligence. These results suggest a promising path for unified visual models in downstream applications such as vision-language-action systems and world models. |
| title | JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generation |
| topic | Graphics Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2605.04128 |