CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Credit
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913162041753600 |
|---|---|
| author | Wang, Kangyu Jiang, Zhiyun Feng, Haibo Zhao, Weijia Liu, Lin Li, Jianguo Lan, Zhenzhong Lin, Weiyao |
| author_facet | Wang, Kangyu Jiang, Zhiyun Feng, Haibo Zhao, Weijia Liu, Lin Li, Jianguo Lan, Zhenzhong Lin, Weiyao |
| contents | Diffusion large language models (dLLMs) generate text through iterative denoising. In commonly adopted parallel decoding schemes, each step confirms only high-confidence positions while remasking the others. By analyzing dLLM denoising traces, we uncover a key inefficiency: models often predict the correct target token several steps before its confidence becomes high enough to be decoded. This gap between early prediction and late decoding forces repeated remasking of already-correct tokens, causing redundant iterations and limiting acceleration. To exploit this temporal redundancy, we introduce Trace Credit to quantify a token's decoding potential by accumulating historical evidence. Building on this, we propose CreditDecoding, a training-free parallel decoding method that fuses Trace Credit with current logits to boost the confidence of correct but underconfident tokens, thereby accelerating denoising and improving robustness. On eight benchmarks, CreditDecoding achieves up to 5.48 times speedup with +0.48 accuracy on LLaDA-8B and consistently improves performance across diverse dLLM architectures and parameter scales. It further scales to long contexts and remains orthogonal to mainstream inference optimizations, making it a practical and widely applicable solution. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_06133 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Credit Wang, Kangyu Jiang, Zhiyun Feng, Haibo Zhao, Weijia Liu, Lin Li, Jianguo Lan, Zhenzhong Lin, Weiyao Computation and Language Artificial Intelligence Diffusion large language models (dLLMs) generate text through iterative denoising. In commonly adopted parallel decoding schemes, each step confirms only high-confidence positions while remasking the others. By analyzing dLLM denoising traces, we uncover a key inefficiency: models often predict the correct target token several steps before its confidence becomes high enough to be decoded. This gap between early prediction and late decoding forces repeated remasking of already-correct tokens, causing redundant iterations and limiting acceleration. To exploit this temporal redundancy, we introduce Trace Credit to quantify a token's decoding potential by accumulating historical evidence. Building on this, we propose CreditDecoding, a training-free parallel decoding method that fuses Trace Credit with current logits to boost the confidence of correct but underconfident tokens, thereby accelerating denoising and improving robustness. On eight benchmarks, CreditDecoding achieves up to 5.48 times speedup with +0.48 accuracy on LLaDA-8B and consistently improves performance across diverse dLLM architectures and parameter scales. It further scales to long contexts and remains orthogonal to mainstream inference optimizations, making it a practical and widely applicable solution. |
| title | CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Credit |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2510.06133 |