Towards Effective Experiential Learning: Dual Guidance for Utilization and Internalization
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917361112580096 |
|---|---|
| author | Bai, Fei Chen, Zhipeng Hao, Chuan Yang, Ming Tao, Ran Dai, Bryan Zhao, Wayne Xin Yang, Jian Xu, Hongteng |
| author_facet | Bai, Fei Chen, Zhipeng Hao, Chuan Yang, Ming Tao, Ran Dai, Bryan Zhao, Wayne Xin Yang, Jian Xu, Hongteng |
| contents | Recently, reinforcement learning~(RL) has become an important approach for improving the capabilities of large language models~(LLMs). In particular, reinforcement learning from verifiable rewards~(RLVR) has emerged as a promising paradigm for reasoning tasks. However, existing RL-based training still remains only a rough approximation to human learning. Human learners leverage both external and internal experience to guide exploration and gradually internalize useful trajectories into stable knowledge. Motivated by this gap, we ask: how can LLMs better utilize and internalize experience during RLVR training? To answer this question, we propose \textbf{D}ual \textbf{G}uidance \textbf{O}ptimization~(\textbf{DGO}), a unified framework that leverages \emph{external} and \emph{internal experience} to improve training effectiveness. Specifically, DGO first constructs an experience bank from previously explored trajectories. The policy then performs exploration under the joint guidance of the experience bank and the model's internal knowledge. The resulting trajectories are further used to refine the experience bank and optimize model parameters, forming a closed loop of experience utilization and internalization. Experiments show that DGO consistently outperforms baseline methods, suggesting that better utilization and internalization of experience lead to more effective reasoning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_24093 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Towards Effective Experiential Learning: Dual Guidance for Utilization and Internalization Bai, Fei Chen, Zhipeng Hao, Chuan Yang, Ming Tao, Ran Dai, Bryan Zhao, Wayne Xin Yang, Jian Xu, Hongteng Machine Learning Artificial Intelligence Recently, reinforcement learning~(RL) has become an important approach for improving the capabilities of large language models~(LLMs). In particular, reinforcement learning from verifiable rewards~(RLVR) has emerged as a promising paradigm for reasoning tasks. However, existing RL-based training still remains only a rough approximation to human learning. Human learners leverage both external and internal experience to guide exploration and gradually internalize useful trajectories into stable knowledge. Motivated by this gap, we ask: how can LLMs better utilize and internalize experience during RLVR training? To answer this question, we propose \textbf{D}ual \textbf{G}uidance \textbf{O}ptimization~(\textbf{DGO}), a unified framework that leverages \emph{external} and \emph{internal experience} to improve training effectiveness. Specifically, DGO first constructs an experience bank from previously explored trajectories. The policy then performs exploration under the joint guidance of the experience bank and the model's internal knowledge. The resulting trajectories are further used to refine the experience bank and optimize model parameters, forming a closed loop of experience utilization and internalization. Experiments show that DGO consistently outperforms baseline methods, suggesting that better utilization and internalization of experience lead to more effective reasoning. |
| title | Towards Effective Experiential Learning: Dual Guidance for Utilization and Internalization |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2603.24093 |