GIFT: Reconciling Post-Training Objectives via Finite-Temperature Gibbs Initialization
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910056967045120 |
|---|---|
| author | Zhao, Zhengyang Ma, Lu Jiang, Yizhen Ma, Xiaochen Meng, Zimo Shen, Chengyu Tang, Lexiang Sun, Haoze Pei, Peng Zhang, Wentao |
| author_facet | Zhao, Zhengyang Ma, Lu Jiang, Yizhen Ma, Xiaochen Meng, Zimo Shen, Chengyu Tang, Lexiang Sun, Haoze Pei, Peng Zhang, Wentao |
| contents | The prevailing post-training paradigm for Large Reasoning Models (LRMs) - Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL) - suffers from an intrinsic optimization mismatch: the rigid supervision inherent in SFT induces distributional collapse, thereby exhausting the exploration space necessary for subsequent RL. In this paper, we reformulate SFT to reconcile post-training objectives and propose Gibbs Initialization with Finite Temperature (GIFT). We characterize standard SFT as a degenerate zero-temperature limit that suppresses base priors. Conversely, GIFT incorporates supervision as a finite-temperature energy potential, establishing a distributional bridge that promotes objective consistency throughout the post-training pipeline. Our experiments demonstrate that GIFT significantly outperforms standard SFT and other competitive baselines when utilized for RL initialization, providing a mathematically principled pathway to preserve exploration and align the two post-training stages. Our code is available at https://github.com/zzy1127/GIFT. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_09233 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | GIFT: Reconciling Post-Training Objectives via Finite-Temperature Gibbs Initialization Zhao, Zhengyang Ma, Lu Jiang, Yizhen Ma, Xiaochen Meng, Zimo Shen, Chengyu Tang, Lexiang Sun, Haoze Pei, Peng Zhang, Wentao Machine Learning Artificial Intelligence Computation and Language The prevailing post-training paradigm for Large Reasoning Models (LRMs) - Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL) - suffers from an intrinsic optimization mismatch: the rigid supervision inherent in SFT induces distributional collapse, thereby exhausting the exploration space necessary for subsequent RL. In this paper, we reformulate SFT to reconcile post-training objectives and propose Gibbs Initialization with Finite Temperature (GIFT). We characterize standard SFT as a degenerate zero-temperature limit that suppresses base priors. Conversely, GIFT incorporates supervision as a finite-temperature energy potential, establishing a distributional bridge that promotes objective consistency throughout the post-training pipeline. Our experiments demonstrate that GIFT significantly outperforms standard SFT and other competitive baselines when utilized for RL initialization, providing a mathematically principled pathway to preserve exploration and align the two post-training stages. Our code is available at https://github.com/zzy1127/GIFT. |
| title | GIFT: Reconciling Post-Training Objectives via Finite-Temperature Gibbs Initialization |
| topic | Machine Learning Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2601.09233 |