PACER: Blockwise Pre-verification for Speculative Decoding with Adaptive Length
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914299651293184 |
|---|---|
| author | Zhang, Situo Zhang, Yifan Zhu, Zichen Wang, Hankun Ma, Da Zhang, Danyang Chen, Lu Yu, Kai |
| author_facet | Zhang, Situo Zhang, Yifan Zhu, Zichen Wang, Hankun Ma, Da Zhang, Danyang Chen, Lu Yu, Kai |
| contents | Speculative decoding (SD) is a powerful technique for accelerating the inference process of large language models (LLMs) without sacrificing accuracy. Typically, SD employs a small draft model to generate a fixed number of draft tokens, which are then verified in parallel by the target model. However, our experiments reveal that the optimal draft length varies significantly across different decoding steps. This variation suggests that using a fixed draft length limits the potential for further improvements in decoding speed. To address this challenge, we propose Pacer, a novel approach that dynamically controls draft length using a lightweight, trainable pre-verification layer. This layer pre-verifies draft tokens blockwise before they are sent to the target model, allowing the draft model to stop token generation if the blockwise pre-verification fails. We implement Pacer on multiple SD model pairs and evaluate its performance across various benchmarks. Our results demonstrate that Pacer achieves up to 2.66x Speedup over autoregressive decoding and consistently outperforms standard speculative decoding. Furthermore, when integrated with Ouroboros, Pacer attains up to 3.09x Speedup. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_01274 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | PACER: Blockwise Pre-verification for Speculative Decoding with Adaptive Length Zhang, Situo Zhang, Yifan Zhu, Zichen Wang, Hankun Ma, Da Zhang, Danyang Chen, Lu Yu, Kai Computation and Language Artificial Intelligence Speculative decoding (SD) is a powerful technique for accelerating the inference process of large language models (LLMs) without sacrificing accuracy. Typically, SD employs a small draft model to generate a fixed number of draft tokens, which are then verified in parallel by the target model. However, our experiments reveal that the optimal draft length varies significantly across different decoding steps. This variation suggests that using a fixed draft length limits the potential for further improvements in decoding speed. To address this challenge, we propose Pacer, a novel approach that dynamically controls draft length using a lightweight, trainable pre-verification layer. This layer pre-verifies draft tokens blockwise before they are sent to the target model, allowing the draft model to stop token generation if the blockwise pre-verification fails. We implement Pacer on multiple SD model pairs and evaluate its performance across various benchmarks. Our results demonstrate that Pacer achieves up to 2.66x Speedup over autoregressive decoding and consistently outperforms standard speculative decoding. Furthermore, when integrated with Ouroboros, Pacer attains up to 3.09x Speedup. |
| title | PACER: Blockwise Pre-verification for Speculative Decoding with Adaptive Length |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2602.01274 |