PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917499180679168 |
|---|---|
| author | Sun, Shengyin Li, Yiming Liu, Renxi Li, Xinqi Zhen, Hui-Ling Lin, Weizhe Chen, Chen Yu, Xianzhi Yuan, Mingxuan Ma, Chen |
| author_facet | Sun, Shengyin Li, Yiming Liu, Renxi Li, Xinqi Zhen, Hui-Ling Lin, Weizhe Chen, Chen Yu, Xianzhi Yuan, Mingxuan Ma, Chen |
| contents | Diffusion large language models (dLLMs) generate text by iteratively denoising masked token sequences. Although dLLMs can predict all masked positions in parallel within each step, the large number of denoising iterations still makes inference expensive. This cost can be reduced spatially by unmasking multiple tokens per step, or temporally by collapsing multiple denoising steps into one verification call. We propose Parallel Speculative Decoding (PSD), a training-free framework that jointly improves inference along both axes. Using the confidence scores from a single forward pass, PSD selects positions to unmask via a configurable, adaptive unmasking policy and constructs multi-depth speculative drafts without extra model calls. A final batched verification pass then applies hierarchical acceptance, keeping the deepest draft that remains consistent with the updated predictions. Experiments on three dLLMs across reasoning and code generation tasks show that PSD achieves favorable trade-offs between inference efficiency and generation quality, reaching up to $5.5\times$ tokens per forward pass with accuracy comparable to greedy decoding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_15609 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding Sun, Shengyin Li, Yiming Liu, Renxi Li, Xinqi Zhen, Hui-Ling Lin, Weizhe Chen, Chen Yu, Xianzhi Yuan, Mingxuan Ma, Chen Computation and Language Diffusion large language models (dLLMs) generate text by iteratively denoising masked token sequences. Although dLLMs can predict all masked positions in parallel within each step, the large number of denoising iterations still makes inference expensive. This cost can be reduced spatially by unmasking multiple tokens per step, or temporally by collapsing multiple denoising steps into one verification call. We propose Parallel Speculative Decoding (PSD), a training-free framework that jointly improves inference along both axes. Using the confidence scores from a single forward pass, PSD selects positions to unmask via a configurable, adaptive unmasking policy and constructs multi-depth speculative drafts without extra model calls. A final batched verification pass then applies hierarchical acceptance, keeping the deepest draft that remains consistent with the updated predictions. Experiments on three dLLMs across reasoning and code generation tasks show that PSD achieves favorable trade-offs between inference efficiency and generation quality, reaching up to $5.5\times$ tokens per forward pass with accuracy comparable to greedy decoding. |
| title | PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2605.15609 |