Taming the Thinker: Conditional Entropy Shaping for Adaptive LLM Reasoning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917510204358656 |
|---|---|
| author | Wei, Shuyu Sun, Jian Qiu, Delai Wang, Yining Liu, Shengping Liang, Jiaen Fu, Ying Huang, Wei Sang, Jitao |
| author_facet | Wei, Shuyu Sun, Jian Qiu, Delai Wang, Yining Liu, Shengping Liang, Jiaen Fu, Ying Huang, Wei Sang, Jitao |
| contents | Entropy-based deep reasoning has emerged as a promising direction for improving the reasoning capabilities of Large Language Models (LLMs), but existing methods often either increase response length indiscriminately or shorten responses at the cost of accuracy. To better balance this trade-off, we introduce Conditional Entropy Shaping (CES), a framework that dynamically controls token-level response entropy, enabling LLMs to produce concise solutions on simple problems while encouraging deeper exploration on hard ones. Built on DAPO, CES uses token-level entropy as an uncertainty signal and applies a conditional bidirectional policy: it penalizes high-entropy "forking point" tokens on correct reasoning paths to improve conciseness, and rewards them on incorrect paths to encourage exploration and error correction. We implement CES on DeepSeek-R1-Distill-7B and evaluate it on 12 mathematical benchmarks. CES consistently improves average accuracy while reducing response length relative to DAPO, and supplementary experiments show similar trends on a smaller 1.5B backbone and on out-of-domain benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_19358 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Taming the Thinker: Conditional Entropy Shaping for Adaptive LLM Reasoning Wei, Shuyu Sun, Jian Qiu, Delai Wang, Yining Liu, Shengping Liang, Jiaen Fu, Ying Huang, Wei Sang, Jitao Computation and Language Entropy-based deep reasoning has emerged as a promising direction for improving the reasoning capabilities of Large Language Models (LLMs), but existing methods often either increase response length indiscriminately or shorten responses at the cost of accuracy. To better balance this trade-off, we introduce Conditional Entropy Shaping (CES), a framework that dynamically controls token-level response entropy, enabling LLMs to produce concise solutions on simple problems while encouraging deeper exploration on hard ones. Built on DAPO, CES uses token-level entropy as an uncertainty signal and applies a conditional bidirectional policy: it penalizes high-entropy "forking point" tokens on correct reasoning paths to improve conciseness, and rewards them on incorrect paths to encourage exploration and error correction. We implement CES on DeepSeek-R1-Distill-7B and evaluate it on 12 mathematical benchmarks. CES consistently improves average accuracy while reducing response length relative to DAPO, and supplementary experiments show similar trends on a smaller 1.5B backbone and on out-of-domain benchmarks. |
| title | Taming the Thinker: Conditional Entropy Shaping for Adaptive LLM Reasoning |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2605.19358 |