DyLLM: Efficient Diffusion LLM Inference via Saliency-based Token Selection and Partial Attention
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914620090875904 |
|---|---|
| author | Lee, Younjoo Dan, Seungkyun Lee, Junghoo Park, Jaiyoung Ahn, Jung Ho |
| author_facet | Lee, Younjoo Dan, Seungkyun Lee, Junghoo Park, Jaiyoung Ahn, Jung Ho |
| contents | Masked diffusion language models enable parallel token decoding, providing a promising alternative to the sequential nature of autoregressive generation. However, their iterative denoising process remains computationally expensive because it repeatedly processes the entire sequence at every step. We observe that across these diffusion steps, most token representations remain stable; only a small subset, which we term salient tokens, contributes meaningfully to the next update. Leveraging this temporal sparsity, we present DyLLM, a training-free inference framework that accelerates decoding by selectively computing only these salient tokens. DyLLM identifies saliency by measuring the cosine similarity of attention contexts between adjacent denoising steps. It recomputes feed-forward and attention operations only for salient tokens while reusing cached activations for the remainder. Across diverse reasoning and code-generation benchmarks, DyLLM achieves up to 9.6x higher throughput while largely preserving the baseline accuracy of representative open-source diffusion LLMs, LLaDA, and Dream. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_08026 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | DyLLM: Efficient Diffusion LLM Inference via Saliency-based Token Selection and Partial Attention Lee, Younjoo Dan, Seungkyun Lee, Junghoo Park, Jaiyoung Ahn, Jung Ho Computation and Language Artificial Intelligence Performance Masked diffusion language models enable parallel token decoding, providing a promising alternative to the sequential nature of autoregressive generation. However, their iterative denoising process remains computationally expensive because it repeatedly processes the entire sequence at every step. We observe that across these diffusion steps, most token representations remain stable; only a small subset, which we term salient tokens, contributes meaningfully to the next update. Leveraging this temporal sparsity, we present DyLLM, a training-free inference framework that accelerates decoding by selectively computing only these salient tokens. DyLLM identifies saliency by measuring the cosine similarity of attention contexts between adjacent denoising steps. It recomputes feed-forward and attention operations only for salient tokens while reusing cached activations for the remainder. Across diverse reasoning and code-generation benchmarks, DyLLM achieves up to 9.6x higher throughput while largely preserving the baseline accuracy of representative open-source diffusion LLMs, LLaDA, and Dream. |
| title | DyLLM: Efficient Diffusion LLM Inference via Saliency-based Token Selection and Partial Attention |
| topic | Computation and Language Artificial Intelligence Performance |
| url | https://arxiv.org/abs/2603.08026 |