Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.09195 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915982752088064 |
|---|---|
| author | Liu, Tao Wu, Taiqiang Yang, Runming Sun, Shaoning Wang, Junjie Yang, Yujiu |
| author_facet | Liu, Tao Wu, Taiqiang Yang, Runming Sun, Shaoning Wang, Junjie Yang, Yujiu |
| contents | Supervised fine-tuning (SFT) is a fundamental post-training strategy to align Large Language Models (LLMs) with human intent. However, traditional SFT often ignores the one-to-many nature of language by forcing alignment with a single reference answer, leading to the model overfitting to non-core expressions. Although our empirical analysis suggests that introducing multiple reference answers can mitigate this issue, the prohibitive data and computational costs necessitate a strategic shift: prioritizing the mitigation of single-reference overfitting over the costly pursuit of answer diversity. To achieve this, we reveal the intrinsic connection between token probability and semantic importance: high-probability tokens carry the core logical framework, while low-probability tokens are mostly replaceable expressions. Based on this insight, we propose ProFit, which selectively masks low-probability tokens to prevent surface-level overfitting. Extensive experiments confirm that ProFit consistently outperforms traditional SFT baselines on general reasoning and mathematical benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_09195 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selection Liu, Tao Wu, Taiqiang Yang, Runming Sun, Shaoning Wang, Junjie Yang, Yujiu Computation and Language Artificial Intelligence Supervised fine-tuning (SFT) is a fundamental post-training strategy to align Large Language Models (LLMs) with human intent. However, traditional SFT often ignores the one-to-many nature of language by forcing alignment with a single reference answer, leading to the model overfitting to non-core expressions. Although our empirical analysis suggests that introducing multiple reference answers can mitigate this issue, the prohibitive data and computational costs necessitate a strategic shift: prioritizing the mitigation of single-reference overfitting over the costly pursuit of answer diversity. To achieve this, we reveal the intrinsic connection between token probability and semantic importance: high-probability tokens carry the core logical framework, while low-probability tokens are mostly replaceable expressions. Based on this insight, we propose ProFit, which selectively masks low-probability tokens to prevent surface-level overfitting. Extensive experiments confirm that ProFit consistently outperforms traditional SFT baselines on general reasoning and mathematical benchmarks. |
| title | ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selection |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2601.09195 |