Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2601.03769 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915722177806336 |
|---|---|
| author | Li, Zihang Wang, Yuhang Zong, Yikun Yu, Wenhan Yuan, Xiaokun Jiang, Runhan Liu, Zirui Yang, Tong Jiang, Arthur |
| author_facet | Li, Zihang Wang, Yuhang Zong, Yikun Yu, Wenhan Yuan, Xiaokun Jiang, Runhan Liu, Zirui Yang, Tong Jiang, Arthur |
| contents | Chain-of-Thought (CoT) prompting has significantly enhanced the mathematical reasoning capabilities of Large Language Models. We find existing fine-tuning datasets frequently suffer from the "answer right but reasoning wrong" probelm, where correct final answers are derived from hallucinated, redundant, or logically invalid intermediate steps. This paper proposes EntroCoT, a unified framework for automatically identifying and refining low-quality CoT supervision traces. EntroCoT first proposes an entropy-based mechanism to segment the reasoning trace into multiple steps at uncertain junctures, and then introduces a Monte Carlo rollout-based mechanism to evaluate the marginal contribution of each step. By accurately filtering deceptive reasoning samples, EntroCoT constructs a high-quality dataset where every intermediate step in each reasoning trace facilitates the final answer. Extensive experiments on mathematical benchmarks demonstrate that fine-tuning on the subset constructed by EntroCoT consistently outperforms the baseslines of full-dataset supervision. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_03769 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | EntroCoT: Enhancing Chain-of-Thought via Adaptive Entropy-Guided Segmentation Li, Zihang Wang, Yuhang Zong, Yikun Yu, Wenhan Yuan, Xiaokun Jiang, Runhan Liu, Zirui Yang, Tong Jiang, Arthur Artificial Intelligence Chain-of-Thought (CoT) prompting has significantly enhanced the mathematical reasoning capabilities of Large Language Models. We find existing fine-tuning datasets frequently suffer from the "answer right but reasoning wrong" probelm, where correct final answers are derived from hallucinated, redundant, or logically invalid intermediate steps. This paper proposes EntroCoT, a unified framework for automatically identifying and refining low-quality CoT supervision traces. EntroCoT first proposes an entropy-based mechanism to segment the reasoning trace into multiple steps at uncertain junctures, and then introduces a Monte Carlo rollout-based mechanism to evaluate the marginal contribution of each step. By accurately filtering deceptive reasoning samples, EntroCoT constructs a high-quality dataset where every intermediate step in each reasoning trace facilitates the final answer. Extensive experiments on mathematical benchmarks demonstrate that fine-tuning on the subset constructed by EntroCoT consistently outperforms the baseslines of full-dataset supervision. |
| title | EntroCoT: Enhancing Chain-of-Thought via Adaptive Entropy-Guided Segmentation |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2601.03769 |