Sparse-LaViDa: Sparse Multimodal Discrete Diffusion Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908714474143744 |
|---|---|
| author | Li, Shufan Gu, Jiuxiang Liu, Kangning Lin, Zhe Wei, Zijun Grover, Aditya Kuen, Jason |
| author_facet | Li, Shufan Gu, Jiuxiang Liu, Kangning Lin, Zhe Wei, Zijun Grover, Aditya Kuen, Jason |
| contents | Masked Discrete Diffusion Models (MDMs) have achieved strong performance across a wide range of multimodal tasks, including image understanding, generation, and editing. However, their inference speed remains suboptimal due to the need to repeatedly process redundant masked tokens at every sampling step. In this work, we propose Sparse-LaViDa, a novel modeling framework that dynamically truncates unnecessary masked tokens at each inference step to accelerate MDM sampling. To preserve generation quality, we introduce specialized register tokens that serve as compact representations for the truncated tokens. Furthermore, to ensure consistency between training and inference, we design a specialized attention mask that faithfully matches the truncated sampling procedure during training. Built upon the state-of-the-art unified MDM LaViDa-O, Sparse-LaViDa achieves up to a 2x speedup across diverse tasks including text-to-image generation, image editing, and mathematical reasoning, while maintaining generation quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_14008 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Sparse-LaViDa: Sparse Multimodal Discrete Diffusion Language Models Li, Shufan Gu, Jiuxiang Liu, Kangning Lin, Zhe Wei, Zijun Grover, Aditya Kuen, Jason Computer Vision and Pattern Recognition Masked Discrete Diffusion Models (MDMs) have achieved strong performance across a wide range of multimodal tasks, including image understanding, generation, and editing. However, their inference speed remains suboptimal due to the need to repeatedly process redundant masked tokens at every sampling step. In this work, we propose Sparse-LaViDa, a novel modeling framework that dynamically truncates unnecessary masked tokens at each inference step to accelerate MDM sampling. To preserve generation quality, we introduce specialized register tokens that serve as compact representations for the truncated tokens. Furthermore, to ensure consistency between training and inference, we design a specialized attention mask that faithfully matches the truncated sampling procedure during training. Built upon the state-of-the-art unified MDM LaViDa-O, Sparse-LaViDa achieves up to a 2x speedup across diverse tasks including text-to-image generation, image editing, and mathematical reasoning, while maintaining generation quality. |
| title | Sparse-LaViDa: Sparse Multimodal Discrete Diffusion Language Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.14008 |