Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.12040 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915736960630784 |
|---|---|
| author | da Luz, Murilo Brandão, Bruno Martins, Luana Oliveira, Gustavo de Oliveira, Bryan Melo, Luckeciano Soares, Telma |
| author_facet | da Luz, Murilo Brandão, Bruno Martins, Luana Oliveira, Gustavo de Oliveira, Bryan Melo, Luckeciano Soares, Telma |
| contents | The use of Large Language Models (LLMs) for reasoning and planning tasks has drawn increasing attention in Artificial Intelligence research. Despite their remarkable progress, these models still exhibit limitations in multi-step inference scenarios, particularly in mathematical and logical reasoning. We introduce PREGU (Partial Reasoning Guided by Uncertainty). PREGU monitors the entropy of the output distribution during autoregressive generation and halts the process whenever entropy exceeds a defined threshold, signaling uncertainty. From that point, a localized search is performed in the latent space to refine the partial reasoning and select the most coherent answer, using the Soft Reasoning method. Experiments conducted with LLaMA-3-8B, Mistral-7B, and Qwen2-7B across four reasoning benchmarks (GSM8K, GSM-Hard, SVAMP, and StrategyQA) showed performance greater than or similar to Soft Reasoning, indicating that entropy can serve as an effective signal to trigger selective refinement during reasoning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_12040 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Partial Reasoning in Language Models: Search and Refinement Guided by Uncertainty da Luz, Murilo Brandão, Bruno Martins, Luana Oliveira, Gustavo de Oliveira, Bryan Melo, Luckeciano Soares, Telma Artificial Intelligence The use of Large Language Models (LLMs) for reasoning and planning tasks has drawn increasing attention in Artificial Intelligence research. Despite their remarkable progress, these models still exhibit limitations in multi-step inference scenarios, particularly in mathematical and logical reasoning. We introduce PREGU (Partial Reasoning Guided by Uncertainty). PREGU monitors the entropy of the output distribution during autoregressive generation and halts the process whenever entropy exceeds a defined threshold, signaling uncertainty. From that point, a localized search is performed in the latent space to refine the partial reasoning and select the most coherent answer, using the Soft Reasoning method. Experiments conducted with LLaMA-3-8B, Mistral-7B, and Qwen2-7B across four reasoning benchmarks (GSM8K, GSM-Hard, SVAMP, and StrategyQA) showed performance greater than or similar to Soft Reasoning, indicating that entropy can serve as an effective signal to trigger selective refinement during reasoning. |
| title | Partial Reasoning in Language Models: Search and Refinement Guided by Uncertainty |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2601.12040 |