Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2506.01480 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909862032572416 |
|---|---|
| author | Pan, Kaihang Wu, Yang Bu, Wendong Shen, Kai Li, Juncheng Wang, Yingting Li, Yunfei Tang, Siliang Xiao, Jun Wu, Fei Zhao, Hang Zhuang, Yueting |
| author_facet | Pan, Kaihang Wu, Yang Bu, Wendong Shen, Kai Li, Juncheng Wang, Yingting Li, Yunfei Tang, Siliang Xiao, Jun Wu, Fei Zhao, Hang Zhuang, Yueting |
| contents | Recent endeavors in Multimodal Large Language Models (MLLMs) aim to unify visual comprehension and generation. However, these two capabilities remain largely independent, as if they are two separate functions encapsulated within the same model. Consequently, visual comprehension does not enhance visual generation, and the reasoning mechanisms of LLMs have not been fully integrated to revolutionize image generation. In this paper, we propose to enable the collaborative co-evolution of visual comprehension and generation, advancing image generation into an iterative introspective process. We introduce a two-stage training approach: supervised fine-tuning teaches the MLLM with the foundational ability to generate genuine CoT for visual generation, while reinforcement learning activates its full potential via an exploration-exploitation trade-off. Ultimately, we unlock the Aha moment in visual generation, advancing MLLMs from text-to-image tasks to unified image generation. Extensive experiments demonstrate that our model not only excels in text-to-image generation and image editing, but also functions as a superior image semantic evaluator with enhanced visual comprehension capabilities. Project Page: https://janus-pro-r1.github.io. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_01480 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Janus-Pro-R1: Advancing Collaborative Visual Comprehension and Generation via Reinforcement Learning Pan, Kaihang Wu, Yang Bu, Wendong Shen, Kai Li, Juncheng Wang, Yingting Li, Yunfei Tang, Siliang Xiao, Jun Wu, Fei Zhao, Hang Zhuang, Yueting Computer Vision and Pattern Recognition Recent endeavors in Multimodal Large Language Models (MLLMs) aim to unify visual comprehension and generation. However, these two capabilities remain largely independent, as if they are two separate functions encapsulated within the same model. Consequently, visual comprehension does not enhance visual generation, and the reasoning mechanisms of LLMs have not been fully integrated to revolutionize image generation. In this paper, we propose to enable the collaborative co-evolution of visual comprehension and generation, advancing image generation into an iterative introspective process. We introduce a two-stage training approach: supervised fine-tuning teaches the MLLM with the foundational ability to generate genuine CoT for visual generation, while reinforcement learning activates its full potential via an exploration-exploitation trade-off. Ultimately, we unlock the Aha moment in visual generation, advancing MLLMs from text-to-image tasks to unified image generation. Extensive experiments demonstrate that our model not only excels in text-to-image generation and image editing, but also functions as a superior image semantic evaluator with enhanced visual comprehension capabilities. Project Page: https://janus-pro-r1.github.io. |
| title | Janus-Pro-R1: Advancing Collaborative Visual Comprehension and Generation via Reinforcement Learning |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2506.01480 |