Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.20500 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918461767155712 |
|---|---|
| author | Li, Xueyan Zenn, Johannes Fadeeva, Ekaterina Su, Guinan Sachan, Mrinmaya Geiping, Jonas |
| author_facet | Li, Xueyan Zenn, Johannes Fadeeva, Ekaterina Su, Guinan Sachan, Mrinmaya Geiping, Jonas |
| contents | Self-consistency boosts inference-time performance by sampling multiple reasoning traces in parallel and voting. However, in constrained domains like math and code, this strategy is compute-inefficient because it samples with replacement, repeatedly revisiting the same high-probability prefixes and duplicate completions. We propose Distinct Leaf Enumeration (DLE), a deterministic decoding method that treats truncated sampling as traversal of a pruned decoding tree and systematically enumerates distinct leaves instead of sampling with replacement. This strategy improves inference efficiency in two ways. Algorithmically, it increases coverage of the truncated search space under a fixed budget by exploring previously unvisited high-probability branches. Systemically, it reuses shared prefixes and reduces redundant token generation. Empirically, DLE explores higher-quality reasoning traces than stochastic self-consistency, yielding better performance on math, coding, and general reasoning tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_20500 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Efficient Test-Time Inference via Deterministic Exploration of Truncated Decoding Trees Li, Xueyan Zenn, Johannes Fadeeva, Ekaterina Su, Guinan Sachan, Mrinmaya Geiping, Jonas Machine Learning Self-consistency boosts inference-time performance by sampling multiple reasoning traces in parallel and voting. However, in constrained domains like math and code, this strategy is compute-inefficient because it samples with replacement, repeatedly revisiting the same high-probability prefixes and duplicate completions. We propose Distinct Leaf Enumeration (DLE), a deterministic decoding method that treats truncated sampling as traversal of a pruned decoding tree and systematically enumerates distinct leaves instead of sampling with replacement. This strategy improves inference efficiency in two ways. Algorithmically, it increases coverage of the truncated search space under a fixed budget by exploring previously unvisited high-probability branches. Systemically, it reuses shared prefixes and reduces redundant token generation. Empirically, DLE explores higher-quality reasoning traces than stochastic self-consistency, yielding better performance on math, coding, and general reasoning tasks. |
| title | Efficient Test-Time Inference via Deterministic Exploration of Truncated Decoding Trees |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2604.20500 |