Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2603.12793 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915859126026240 |
|---|---|
| author | Zhang, Yichen Peng, Da Guo, Zonghao Zhang, Zijian Yang, Xuesong Sun, Tong Sun, Shichu Zhang, Yidan Li, Yanghao Zhao, Haiyan Xu, Wang Shi, Qi Sun, Yangang Chen, Chi Wang, Shuo Yan, Yukun Han, Xu Ma, Qiang Ke, Wei Wang, Liang Liu, Zhiyuan Sun, Maosong |
| author_facet | Zhang, Yichen Peng, Da Guo, Zonghao Zhang, Zijian Yang, Xuesong Sun, Tong Sun, Shichu Zhang, Yidan Li, Yanghao Zhao, Haiyan Xu, Wang Shi, Qi Sun, Yangang Chen, Chi Wang, Shuo Yan, Yukun Han, Xu Ma, Qiang Ke, Wei Wang, Liang Liu, Zhiyuan Sun, Maosong |
| contents | A recent cutting-edge topic in multimodal modeling is to unify visual comprehension and generation within a single model. However, the two tasks demand mismatched decoding regimes and visual representations, making it non-trivial to jointly optimize within a shared feature space. In this work, we present Cheers, a unified multimodal model that decouples patch-level details from semantic representations, thereby stabilizing semantics for multimodal understanding and improving fidelity for image generation via gated detail residuals. Cheers includes three key components: (i) a unified vision tokenizer that encodes and compresses image latent states into semantic tokens for efficient LLM conditioning, (ii) an LLM-based Transformer that unifies autoregressive decoding for text generation and diffusion decoding for image generation, and (iii) a cascaded flow matching head that decodes visual semantics first and then injects semantically gated detail residuals from the vision tokenizer to refine high-frequency content. Experiments on popular benchmarks demonstrate that Cheers matches or surpasses advanced UMMs in both visual understanding and generation. Cheers also achieves 4x token compression, enabling more efficient high-resolution image encoding and generation. Notably, Cheers outperforms the Tar-1.5B on the popular benchmarks GenEval and MMBench, while requiring only 20% of the training cost, indicating effective and efficient (i.e., 4x token compression) unified multimodal modeling. We will release all code and data for future research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_12793 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Cheers: Decoupling Patch Details from Semantic Representations Enables Unified Multimodal Comprehension and Generation Zhang, Yichen Peng, Da Guo, Zonghao Zhang, Zijian Yang, Xuesong Sun, Tong Sun, Shichu Zhang, Yidan Li, Yanghao Zhao, Haiyan Xu, Wang Shi, Qi Sun, Yangang Chen, Chi Wang, Shuo Yan, Yukun Han, Xu Ma, Qiang Ke, Wei Wang, Liang Liu, Zhiyuan Sun, Maosong Computer Vision and Pattern Recognition Artificial Intelligence 97P80 A recent cutting-edge topic in multimodal modeling is to unify visual comprehension and generation within a single model. However, the two tasks demand mismatched decoding regimes and visual representations, making it non-trivial to jointly optimize within a shared feature space. In this work, we present Cheers, a unified multimodal model that decouples patch-level details from semantic representations, thereby stabilizing semantics for multimodal understanding and improving fidelity for image generation via gated detail residuals. Cheers includes three key components: (i) a unified vision tokenizer that encodes and compresses image latent states into semantic tokens for efficient LLM conditioning, (ii) an LLM-based Transformer that unifies autoregressive decoding for text generation and diffusion decoding for image generation, and (iii) a cascaded flow matching head that decodes visual semantics first and then injects semantically gated detail residuals from the vision tokenizer to refine high-frequency content. Experiments on popular benchmarks demonstrate that Cheers matches or surpasses advanced UMMs in both visual understanding and generation. Cheers also achieves 4x token compression, enabling more efficient high-resolution image encoding and generation. Notably, Cheers outperforms the Tar-1.5B on the popular benchmarks GenEval and MMBench, while requiring only 20% of the training cost, indicating effective and efficient (i.e., 4x token compression) unified multimodal modeling. We will release all code and data for future research. |
| title | Cheers: Decoupling Patch Details from Semantic Representations Enables Unified Multimodal Comprehension and Generation |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence 97P80 |
| url | https://arxiv.org/abs/2603.12793 |