Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.18810 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917509455675392 |
|---|---|
| author | Wu, Tianyu Yao, Yu Qi, Zhenting Zheng, Han Wang, Zhuohan Ma, Haoran Liao, Lawrence Lakkaraju, Himabindu Li, Ju Du, Yilun |
| author_facet | Wu, Tianyu Yao, Yu Qi, Zhenting Zheng, Han Wang, Zhuohan Ma, Haoran Liao, Lawrence Lakkaraju, Himabindu Li, Ju Du, Yilun |
| contents | Speculative decoding accelerates LLM inference by having a small drafter propose tokens that a larger target model verifies in parallel. Recent diffusion-based parallel drafters such as DFlash predict the full B-token block in one forward pass, enabling deeper drafters and longer accepted blocks. However, existing multi-token drafter objectives often use fixed position-dependent weighting schedules, such as head-dependent weights or block-position decays, which do not adapt as the positions limiting acceptance change during training. To address this, we derive per-position training weights from a differentiable surrogate of expected accepted draft length, matching the weight of each position to its log-probability gradient contribution. The resulting loss, D-PACE (Dynamic Position-Aware Cross-Entropy), shifts training signal toward positions that currently limit acceptance as the drafter improves. Across six benchmarks, two Qwen3-4B draft depths, two decoding temperatures, and two additional target models, D-PACE consistently improves both wall-clock speedup and average emitted length, with 2.3\% measured training-time overhead and no changes to the drafter architecture or inference procedure. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_18810 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | D-PACE: Dynamic Position-Aware Cross-Entropy for Parallel Speculative Drafting Wu, Tianyu Yao, Yu Qi, Zhenting Zheng, Han Wang, Zhuohan Ma, Haoran Liao, Lawrence Lakkaraju, Himabindu Li, Ju Du, Yilun Machine Learning Artificial Intelligence Speculative decoding accelerates LLM inference by having a small drafter propose tokens that a larger target model verifies in parallel. Recent diffusion-based parallel drafters such as DFlash predict the full B-token block in one forward pass, enabling deeper drafters and longer accepted blocks. However, existing multi-token drafter objectives often use fixed position-dependent weighting schedules, such as head-dependent weights or block-position decays, which do not adapt as the positions limiting acceptance change during training. To address this, we derive per-position training weights from a differentiable surrogate of expected accepted draft length, matching the weight of each position to its log-probability gradient contribution. The resulting loss, D-PACE (Dynamic Position-Aware Cross-Entropy), shifts training signal toward positions that currently limit acceptance as the drafter improves. Across six benchmarks, two Qwen3-4B draft depths, two decoding temperatures, and two additional target models, D-PACE consistently improves both wall-clock speedup and average emitted length, with 2.3\% measured training-time overhead and no changes to the drafter architecture or inference procedure. |
| title | D-PACE: Dynamic Position-Aware Cross-Entropy for Parallel Speculative Drafting |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2605.18810 |