Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.19278 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917225093398528 |
|---|---|
| author | Liu, Fuliang Li, Xue Zhao, Ketai Gao, Yinxi Zhou, Ziyan Zhang, Zhonghui Wang, Zhibin Dou, Wanchun Zhong, Sheng Tian, Chen |
| author_facet | Liu, Fuliang Li, Xue Zhao, Ketai Gao, Yinxi Zhou, Ziyan Zhang, Zhonghui Wang, Zhibin Dou, Wanchun Zhong, Sheng Tian, Chen |
| contents | Speculative decoding is an effective and lossless approach for accelerating LLM inference. However, existing widely adopted model-based draft designs, such as EAGLE3, improve accuracy at the cost of multi-step autoregressive inference, resulting in high drafting latency and ultimately rendering the drafting stage itself a performance bottleneck. Inspired by diffusion-based large language models (dLLMs), we propose DART, which leverages parallel generation to reduce drafting latency. DART predicts logits for multiple future masked positions in parallel within a single forward pass based on hidden states of the target model, thereby eliminating autoregressive rollouts in the draft model while preserving a lightweight design. Based on these parallel logit predictions, we further introduce an efficient tree pruning algorithm that constructs high-quality draft token trees with N-gram-enforced semantic continuity. DART substantially reduces draft-stage overhead while preserving high draft accuracy, leading to significantly improved end-to-end decoding speed. Experimental results demonstrate that DART achieves a 2.03x--3.44x wall-clock time speedup across multiple datasets, surpassing EAGLE3 by 30% on average and offering a practical speculative decoding framework. Code is released at https://github.com/fvliang/DART. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_19278 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | DART: Diffusion-Inspired Speculative Decoding for Fast LLM Inference Liu, Fuliang Li, Xue Zhao, Ketai Gao, Yinxi Zhou, Ziyan Zhang, Zhonghui Wang, Zhibin Dou, Wanchun Zhong, Sheng Tian, Chen Computation and Language Speculative decoding is an effective and lossless approach for accelerating LLM inference. However, existing widely adopted model-based draft designs, such as EAGLE3, improve accuracy at the cost of multi-step autoregressive inference, resulting in high drafting latency and ultimately rendering the drafting stage itself a performance bottleneck. Inspired by diffusion-based large language models (dLLMs), we propose DART, which leverages parallel generation to reduce drafting latency. DART predicts logits for multiple future masked positions in parallel within a single forward pass based on hidden states of the target model, thereby eliminating autoregressive rollouts in the draft model while preserving a lightweight design. Based on these parallel logit predictions, we further introduce an efficient tree pruning algorithm that constructs high-quality draft token trees with N-gram-enforced semantic continuity. DART substantially reduces draft-stage overhead while preserving high draft accuracy, leading to significantly improved end-to-end decoding speed. Experimental results demonstrate that DART achieves a 2.03x--3.44x wall-clock time speedup across multiple datasets, surpassing EAGLE3 by 30% on average and offering a practical speculative decoding framework. Code is released at https://github.com/fvliang/DART. |
| title | DART: Diffusion-Inspired Speculative Decoding for Fast LLM Inference |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2601.19278 |