Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.10953 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914349416710144 |
|---|---|
| author | Cao, Mingyu Correia, Alvaro H. C. Louizos, Christos Liu, Shiwei Yin, Lu |
| author_facet | Cao, Mingyu Correia, Alvaro H. C. Louizos, Christos Liu, Shiwei Yin, Lu |
| contents | Diffusion Language Models (DLMs) generate text by iteratively denoising a masked sequence, repeatedly deciding which positions to commit at each step. Standard decoding follows a greedy rule: unmask the most confident positions, yet this local choice can lock the model into a suboptimal unmasking order, especially on reasoning-heavy prompts. We present SOAR, a training-free decoding algorithm that adapts its behavior to the model's uncertainty. When confidence is low, SOAR briefly widens the search over alternative unmasking decisions to avoid premature commitments; when confidence is high, it collapses the search and decodes many positions in parallel to reduce the number of denoising iterations. Across mathematical reasoning and code generation benchmarks (GSM8K, MBPP, HumanEval) on Dream-7B and LLaDA-8B, SOAR improves generation quality while maintaining competitive inference speed, offering a practical way to balance quality and efficiency in DLM decoding. Our Code is available at https://github.com/duterscmy/SOAR |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_10953 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Search or Accelerate: Confidence-Switched Position Beam Search for Diffusion Language Models Cao, Mingyu Correia, Alvaro H. C. Louizos, Christos Liu, Shiwei Yin, Lu Computation and Language Artificial Intelligence Diffusion Language Models (DLMs) generate text by iteratively denoising a masked sequence, repeatedly deciding which positions to commit at each step. Standard decoding follows a greedy rule: unmask the most confident positions, yet this local choice can lock the model into a suboptimal unmasking order, especially on reasoning-heavy prompts. We present SOAR, a training-free decoding algorithm that adapts its behavior to the model's uncertainty. When confidence is low, SOAR briefly widens the search over alternative unmasking decisions to avoid premature commitments; when confidence is high, it collapses the search and decodes many positions in parallel to reduce the number of denoising iterations. Across mathematical reasoning and code generation benchmarks (GSM8K, MBPP, HumanEval) on Dream-7B and LLaDA-8B, SOAR improves generation quality while maintaining competitive inference speed, offering a practical way to balance quality and efficiency in DLM decoding. Our Code is available at https://github.com/duterscmy/SOAR |
| title | Search or Accelerate: Confidence-Switched Position Beam Search for Diffusion Language Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2602.10953 |