Muddit: Liberating Generation Beyond Text-to-Image with a Unified Discrete Diffusion Model
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914600576876544 |
|---|---|
| author | Shi, Qingyu Bai, Jinbin Zhao, Zhuoran Chai, Wenhao Yu, Kaidong Wu, Jianzong Tong, Yunhai Li, Xiangtai Li, Xuelong Yan, Shuicheng |
| author_facet | Shi, Qingyu Bai, Jinbin Zhao, Zhuoran Chai, Wenhao Yu, Kaidong Wu, Jianzong Tong, Yunhai Li, Xiangtai Li, Xuelong Yan, Shuicheng |
| contents | Unified generation models aim to handle diverse tasks across modalities -- such as text generation, image generation, and vision-language reasoning -- within a single architecture and decoding paradigm. Autoregressive unified models suffer from slow inference due to sequential decoding, and non-autoregressive unified models suffer from weak generalization due to limited pretrained backbones. We introduce the second-generation Meissonic: Muddit, a unified discrete diffusion transformer that enables fast and parallel generation across both text and image modalities. Unlike prior unified diffusion models trained from scratch, Muddit integrates strong visual priors from a pretrained text-to-image backbone with a lightweight text decoder, enabling flexible and high-quality multimodal generation under a unified architecture. Empirical results show that Muddit achieves competitive or superior performance compared to significantly larger autoregressive models in both quality and efficiency. The work highlights the potential of purely discrete diffusion, when equipped with strong visual priors, as a scalable and effective backbone for unified generation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_23606 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Muddit: Liberating Generation Beyond Text-to-Image with a Unified Discrete Diffusion Model Shi, Qingyu Bai, Jinbin Zhao, Zhuoran Chai, Wenhao Yu, Kaidong Wu, Jianzong Tong, Yunhai Li, Xiangtai Li, Xuelong Yan, Shuicheng Machine Learning Computer Vision and Pattern Recognition Unified generation models aim to handle diverse tasks across modalities -- such as text generation, image generation, and vision-language reasoning -- within a single architecture and decoding paradigm. Autoregressive unified models suffer from slow inference due to sequential decoding, and non-autoregressive unified models suffer from weak generalization due to limited pretrained backbones. We introduce the second-generation Meissonic: Muddit, a unified discrete diffusion transformer that enables fast and parallel generation across both text and image modalities. Unlike prior unified diffusion models trained from scratch, Muddit integrates strong visual priors from a pretrained text-to-image backbone with a lightweight text decoder, enabling flexible and high-quality multimodal generation under a unified architecture. Empirical results show that Muddit achieves competitive or superior performance compared to significantly larger autoregressive models in both quality and efficiency. The work highlights the potential of purely discrete diffusion, when equipped with strong visual priors, as a scalable and effective backbone for unified generation. |
| title | Muddit: Liberating Generation Beyond Text-to-Image with a Unified Discrete Diffusion Model |
| topic | Machine Learning Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.23606 |