Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.10827 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915933723820032 |
|---|---|
| author | Choraria, Moulik Gerogiannis, Argyrios Das, Anirban Chakraborty, Supriyo Kapusuzoglu, Berkcan Lee, Chia-Hsuan Balasubramaniam, Kartik Zhang, Shi-Xiong Sahu, Sambit |
| author_facet | Choraria, Moulik Gerogiannis, Argyrios Das, Anirban Chakraborty, Supriyo Kapusuzoglu, Berkcan Lee, Chia-Hsuan Balasubramaniam, Kartik Zhang, Shi-Xiong Sahu, Sambit |
| contents | Compute scaling for LLM reasoning requires allocating budget between exploring solution approaches ($breadth$) and refining promising solutions ($depth$). Most methods implicitly trade off one for the other, yet why a given trade-off works remains unclear, and validation on a single model obscures the role of the model itself. We argue that $\textbf{the optimal strategy depends on the model's diversity profile, the spread of probability mass across solution approaches, and that this must be characterized before any exploration strategy is adopted.}$ We formalize this through a theoretical framework decomposing reasoning uncertainty and derive conditions under which tree-style depth refinement outperforms parallel sampling. We validate it on Qwen-3 4B and Olmo-3 7B families, showing that lightweight signals suffice for depth-based refinement on low-diversity aligned models while yielding limited utility for high-diversity base models, which we hypothesize require stronger compensation for lower exploration coverage. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_10827 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Your Model Diversity, Not Method, Determines Reasoning Strategy Choraria, Moulik Gerogiannis, Argyrios Das, Anirban Chakraborty, Supriyo Kapusuzoglu, Berkcan Lee, Chia-Hsuan Balasubramaniam, Kartik Zhang, Shi-Xiong Sahu, Sambit Artificial Intelligence Compute scaling for LLM reasoning requires allocating budget between exploring solution approaches ($breadth$) and refining promising solutions ($depth$). Most methods implicitly trade off one for the other, yet why a given trade-off works remains unclear, and validation on a single model obscures the role of the model itself. We argue that $\textbf{the optimal strategy depends on the model's diversity profile, the spread of probability mass across solution approaches, and that this must be characterized before any exploration strategy is adopted.}$ We formalize this through a theoretical framework decomposing reasoning uncertainty and derive conditions under which tree-style depth refinement outperforms parallel sampling. We validate it on Qwen-3 4B and Olmo-3 7B families, showing that lightweight signals suffice for depth-based refinement on low-diversity aligned models while yielding limited utility for high-diversity base models, which we hypothesize require stronger compensation for lower exploration coverage. |
| title | Your Model Diversity, Not Method, Determines Reasoning Strategy |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2604.10827 |