Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.02752 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916987240710144 |
|---|---|
| author | Zhang, Hangfan Xu, Siyuan Guo, Zhimeng Zhu, Huaisheng Liu, Shicheng Wang, Xinrun Zhang, Qiaosheng Chen, Yang Ye, Peng Bai, Lei Hu, Shuyue |
| author_facet | Zhang, Hangfan Xu, Siyuan Guo, Zhimeng Zhu, Huaisheng Liu, Shicheng Wang, Xinrun Zhang, Qiaosheng Chen, Yang Ye, Peng Bai, Lei Hu, Shuyue |
| contents | Reinforcement learning (RL) has demonstrated potential in enhancing the reasoning capabilities of large language models (LLMs), but such training typically demands substantial efforts in creating and annotating data. In this work, we explore improving LLMs through RL with minimal data. Our approach alternates between the LLM proposing a task and then attempting to solve it. To minimize data dependency, we introduce two novel mechanisms grounded in self-awareness: (1) self-aware difficulty prediction, where the model learns to assess task difficulty relative to its own abilities and prioritize challenging yet solvable tasks, and (2) self-aware limit breaking, where the model recognizes when a task is beyond its capability boundary and proactively requests external data to break through that limit. Extensive experiments on nine benchmarks showing a 53.8% relative improvement with less than 1.2% extra data demonstrate the efficacy of self-aware RL and underscore the promise of self-evolving agent training. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_02752 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The Path of Self-Evolving Large Language Models: Achieving Data-Efficient Learning via Intrinsic Feedback Zhang, Hangfan Xu, Siyuan Guo, Zhimeng Zhu, Huaisheng Liu, Shicheng Wang, Xinrun Zhang, Qiaosheng Chen, Yang Ye, Peng Bai, Lei Hu, Shuyue Computation and Language Reinforcement learning (RL) has demonstrated potential in enhancing the reasoning capabilities of large language models (LLMs), but such training typically demands substantial efforts in creating and annotating data. In this work, we explore improving LLMs through RL with minimal data. Our approach alternates between the LLM proposing a task and then attempting to solve it. To minimize data dependency, we introduce two novel mechanisms grounded in self-awareness: (1) self-aware difficulty prediction, where the model learns to assess task difficulty relative to its own abilities and prioritize challenging yet solvable tasks, and (2) self-aware limit breaking, where the model recognizes when a task is beyond its capability boundary and proactively requests external data to break through that limit. Extensive experiments on nine benchmarks showing a 53.8% relative improvement with less than 1.2% extra data demonstrate the efficacy of self-aware RL and underscore the promise of self-evolving agent training. |
| title | The Path of Self-Evolving Large Language Models: Achieving Data-Efficient Learning via Intrinsic Feedback |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2510.02752 |