Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.22389 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914587841921024 |
|---|---|
| author | Li, Xiaoyuan Ma, Yubo Li, Chengpeng Zhu, Fengbin Yu, Yiyao Bao, Keqin Wang, Wenjie Feng, Fuli Liu, Dayiheng |
| author_facet | Li, Xiaoyuan Ma, Yubo Li, Chengpeng Zhu, Fengbin Yu, Yiyao Bao, Keqin Wang, Wenjie Feng, Fuli Liu, Dayiheng |
| contents | Effectively training Large Language Models (LLMs) for complex, long-CoT reasoning is often bottlenecked by the need for massive high-quality reasoning data. Existing methods are either computationally expensive or fail to reliably distinguish high- from low-quality reasoning samples. To address this, we propose High-Entropy Sum (HES), a training-free metric that quantifies reasoning quality by summing only the entropy of the top (e.g., 0.5\%) highest-entropy tokens in each reasoning sample. We validate HES across three mainstream training paradigms: Supervised Fine-tuning (SFT), Rejection Fine-tuning (RFT), and Reinforcement Learning (RL), with extensive results demonstrating its consistent effectiveness and significantly reduced computational overhead. In SFT, training on the top 20\% HES-ranked data matches full-dataset performance, while using the lowest-HES data degrades it. In RFT, our HES-based training approach significantly outperforms baseline methods. In RL, HES-selected successful trajectories enable the model to learn strong reasoning patterns, significantly surpassing other compared methods. Our findings establish HES as a robust, training-free metric that enables a unified, effective, and efficient method for developing advanced reasoning in LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_22389 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Unified Data Selection for LLM Reasoning Li, Xiaoyuan Ma, Yubo Li, Chengpeng Zhu, Fengbin Yu, Yiyao Bao, Keqin Wang, Wenjie Feng, Fuli Liu, Dayiheng Computation and Language Effectively training Large Language Models (LLMs) for complex, long-CoT reasoning is often bottlenecked by the need for massive high-quality reasoning data. Existing methods are either computationally expensive or fail to reliably distinguish high- from low-quality reasoning samples. To address this, we propose High-Entropy Sum (HES), a training-free metric that quantifies reasoning quality by summing only the entropy of the top (e.g., 0.5\%) highest-entropy tokens in each reasoning sample. We validate HES across three mainstream training paradigms: Supervised Fine-tuning (SFT), Rejection Fine-tuning (RFT), and Reinforcement Learning (RL), with extensive results demonstrating its consistent effectiveness and significantly reduced computational overhead. In SFT, training on the top 20\% HES-ranked data matches full-dataset performance, while using the lowest-HES data degrades it. In RFT, our HES-based training approach significantly outperforms baseline methods. In RL, HES-selected successful trajectories enable the model to learn strong reasoning patterns, significantly surpassing other compared methods. Our findings establish HES as a robust, training-free metric that enables a unified, effective, and efficient method for developing advanced reasoning in LLMs. |
| title | Unified Data Selection for LLM Reasoning |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2605.22389 |