Parallel Token Prediction for Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910041367379968 |
|---|---|
| author | Draxler, Felix Will, Justus Sofian, Farrin Marouf Karaletsos, Theofanis Singh, Sameer Mandt, Stephan |
| author_facet | Draxler, Felix Will, Justus Sofian, Farrin Marouf Karaletsos, Theofanis Singh, Sameer Mandt, Stephan |
| contents | Autoregressive decoding in language models is inherently slow, generating only one token per forward pass. We propose Parallel Token Prediction (PTP), a general-purpose framework for predicting multiple tokens in a single model call. PTP moves the source of randomness from post-hoc sampling to random input variables, making future tokens deterministic functions of those inputs and thus jointly predictable in a single forward pass. We prove that a single PTP call can represent arbitrary dependencies between tokens. PTP is trained by distilling an existing model or through inverse autoregressive training without a teacher. Experimentally, PTP achieves a 2.4x speedup on a diverse-task speculative decoding benchmark. We provide code and checkpoints at https://github.com/mandt-lab/ptp. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_21323 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Parallel Token Prediction for Language Models Draxler, Felix Will, Justus Sofian, Farrin Marouf Karaletsos, Theofanis Singh, Sameer Mandt, Stephan Computation and Language Machine Learning Autoregressive decoding in language models is inherently slow, generating only one token per forward pass. We propose Parallel Token Prediction (PTP), a general-purpose framework for predicting multiple tokens in a single model call. PTP moves the source of randomness from post-hoc sampling to random input variables, making future tokens deterministic functions of those inputs and thus jointly predictable in a single forward pass. We prove that a single PTP call can represent arbitrary dependencies between tokens. PTP is trained by distilling an existing model or through inverse autoregressive training without a teacher. Experimentally, PTP achieves a 2.4x speedup on a diverse-task speculative decoding benchmark. We provide code and checkpoints at https://github.com/mandt-lab/ptp. |
| title | Parallel Token Prediction for Language Models |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2512.21323 |