Speculative Decoding for Multi-Sample Inference
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910863486615552 |
|---|---|
| author | Li, Yiwei Shi, Jiayi Feng, Shaoxiong Yuan, Peiwen Wang, Xinglin Zhang, Yueqi Zhang, Ji Tan, Chuyi Pan, Boyuan Hu, Yao Li, Kan |
| author_facet | Li, Yiwei Shi, Jiayi Feng, Shaoxiong Yuan, Peiwen Wang, Xinglin Zhang, Yueqi Zhang, Ji Tan, Chuyi Pan, Boyuan Hu, Yao Li, Kan |
| contents | We propose a novel speculative decoding method tailored for multi-sample reasoning scenarios, such as self-consistency and Best-of-N sampling. Our method exploits the intrinsic consensus of parallel generation paths to synthesize high-quality draft tokens without requiring auxiliary models or external databases. By dynamically analyzing structural patterns across parallel reasoning paths through a probabilistic aggregation mechanism, it identifies consensus token sequences that align with the decoding distribution. Evaluations on mathematical reasoning benchmarks demonstrate a substantial improvement in draft acceptance rates over baselines, while reducing the latency in draft token construction. This work establishes a paradigm shift for efficient multi-sample inference, enabling seamless integration of speculative decoding with sampling-based reasoning techniques. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_05330 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Speculative Decoding for Multi-Sample Inference Li, Yiwei Shi, Jiayi Feng, Shaoxiong Yuan, Peiwen Wang, Xinglin Zhang, Yueqi Zhang, Ji Tan, Chuyi Pan, Boyuan Hu, Yao Li, Kan Computation and Language Artificial Intelligence We propose a novel speculative decoding method tailored for multi-sample reasoning scenarios, such as self-consistency and Best-of-N sampling. Our method exploits the intrinsic consensus of parallel generation paths to synthesize high-quality draft tokens without requiring auxiliary models or external databases. By dynamically analyzing structural patterns across parallel reasoning paths through a probabilistic aggregation mechanism, it identifies consensus token sequences that align with the decoding distribution. Evaluations on mathematical reasoning benchmarks demonstrate a substantial improvement in draft acceptance rates over baselines, while reducing the latency in draft token construction. This work establishes a paradigm shift for efficient multi-sample inference, enabling seamless integration of speculative decoding with sampling-based reasoning techniques. |
| title | Speculative Decoding for Multi-Sample Inference |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2503.05330 |