Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.29620 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918422891200512 |
|---|---|
| author | Chen, Shuang Shou, Quanxin Chen, Hangting Zhou, Yucheng Feng, Kaituo Hu, Wenbo Zhang, Yi-Fan Lin, Yunlong Huang, Wenxuan Song, Mingyang Dai, Dasen Jiang, Bolin Zhang, Manyuan Zhang, Shi-Xue Jiang, Zhengkai Wang, Lucas Zhong, Zhao Cheng, Yu Peng, Nanyun |
| author_facet | Chen, Shuang Shou, Quanxin Chen, Hangting Zhou, Yucheng Feng, Kaituo Hu, Wenbo Zhang, Yi-Fan Lin, Yunlong Huang, Wenxuan Song, Mingyang Dai, Dasen Jiang, Bolin Zhang, Manyuan Zhang, Shi-Xue Jiang, Zhengkai Wang, Lucas Zhong, Zhao Cheng, Yu Peng, Nanyun |
| contents | Unified multimodal models provide a natural and promising architecture for understanding diverse and complex real-world knowledge while generating high-quality images. However, they still rely primarily on frozen parametric knowledge, which makes them struggle with real-world image generation involving long-tail and knowledge-intensive concepts. Inspired by the broad success of agents on real-world tasks, we explore agentic modeling to address this limitation. Specifically, we present Unify-Agent, a unified multimodal agent for world-grounded image synthesis, which reframes image generation as an agentic pipeline consisting of prompt understanding, multimodal evidence searching, grounded recaptioning, and final synthesis. To train our model, we construct a tailored multimodal data pipeline and curate 143K high-quality agent trajectories for world-grounded image synthesis, enabling effective supervision over the full agentic generation process. We further introduce FactIP, a benchmark covering 12 categories of culturally significant and long-tail factual concepts that explicitly requires external knowledge grounding. Extensive experiments show that our proposed Unify-Agent substantially improves over its base unified model across diverse benchmarks and real world generation tasks, while approaching the world knowledge capabilities of the strongest closed-source models. As an early exploration of agent-based modeling for world-grounded image synthesis, our work highlights the value of tightly coupling reasoning, searching, and generation for reliable open-world agentic image synthesis. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_29620 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesis Chen, Shuang Shou, Quanxin Chen, Hangting Zhou, Yucheng Feng, Kaituo Hu, Wenbo Zhang, Yi-Fan Lin, Yunlong Huang, Wenxuan Song, Mingyang Dai, Dasen Jiang, Bolin Zhang, Manyuan Zhang, Shi-Xue Jiang, Zhengkai Wang, Lucas Zhong, Zhao Cheng, Yu Peng, Nanyun Computer Vision and Pattern Recognition Multimedia Unified multimodal models provide a natural and promising architecture for understanding diverse and complex real-world knowledge while generating high-quality images. However, they still rely primarily on frozen parametric knowledge, which makes them struggle with real-world image generation involving long-tail and knowledge-intensive concepts. Inspired by the broad success of agents on real-world tasks, we explore agentic modeling to address this limitation. Specifically, we present Unify-Agent, a unified multimodal agent for world-grounded image synthesis, which reframes image generation as an agentic pipeline consisting of prompt understanding, multimodal evidence searching, grounded recaptioning, and final synthesis. To train our model, we construct a tailored multimodal data pipeline and curate 143K high-quality agent trajectories for world-grounded image synthesis, enabling effective supervision over the full agentic generation process. We further introduce FactIP, a benchmark covering 12 categories of culturally significant and long-tail factual concepts that explicitly requires external knowledge grounding. Extensive experiments show that our proposed Unify-Agent substantially improves over its base unified model across diverse benchmarks and real world generation tasks, while approaching the world knowledge capabilities of the strongest closed-source models. As an early exploration of agent-based modeling for world-grounded image synthesis, our work highlights the value of tightly coupling reasoning, searching, and generation for reliable open-world agentic image synthesis. |
| title | Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesis |
| topic | Computer Vision and Pattern Recognition Multimedia |
| url | https://arxiv.org/abs/2603.29620 |