Self-Rewarding Rubric-Based Reinforcement Learning for Open-Ended Reasoning
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912615980072960 |
|---|---|
| author | Ye, Zhiling Yue, Yun Wang, Haowen Han, Xudong Jiang, Jiadi Wei, Cheng Fan, Lei Liang, Jiaxin Zhang, Shuowen Li, Ji Guo, Chunxiao Wang, Jian Wei, Peng Gu, Jinjie |
| author_facet | Ye, Zhiling Yue, Yun Wang, Haowen Han, Xudong Jiang, Jiadi Wei, Cheng Fan, Lei Liang, Jiaxin Zhang, Shuowen Li, Ji Guo, Chunxiao Wang, Jian Wei, Peng Gu, Jinjie |
| contents | Open-ended evaluation is essential for deploying large language models in real-world settings. In studying HealthBench, we observe that using the model itself as a grader and generating rubric-based reward signals substantially improves reasoning performance. Remarkably, the trained model also becomes a stronger grader. Motivated by this, we introduce Self-Rewarding Rubric-Based Reinforcement Learning for Open-Ended Reasoning, a lightweight framework that enables faster and more resource-efficient training while surpassing baselines. Remarkably, on Qwen3-32B, training with just the 4000-sample HealthBench Easy subset is sufficient to obtain a model that exceeds GPT-5 on HealthBench Hard. Incorporating a small amount of teacher-graded data further enhances performance for less capable models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_25534 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Self-Rewarding Rubric-Based Reinforcement Learning for Open-Ended Reasoning Ye, Zhiling Yue, Yun Wang, Haowen Han, Xudong Jiang, Jiadi Wei, Cheng Fan, Lei Liang, Jiaxin Zhang, Shuowen Li, Ji Guo, Chunxiao Wang, Jian Wei, Peng Gu, Jinjie Computation and Language Artificial Intelligence Machine Learning Open-ended evaluation is essential for deploying large language models in real-world settings. In studying HealthBench, we observe that using the model itself as a grader and generating rubric-based reward signals substantially improves reasoning performance. Remarkably, the trained model also becomes a stronger grader. Motivated by this, we introduce Self-Rewarding Rubric-Based Reinforcement Learning for Open-Ended Reasoning, a lightweight framework that enables faster and more resource-efficient training while surpassing baselines. Remarkably, on Qwen3-32B, training with just the 4000-sample HealthBench Easy subset is sufficient to obtain a model that exceeds GPT-5 on HealthBench Hard. Incorporating a small amount of teacher-graded data further enhances performance for less capable models. |
| title | Self-Rewarding Rubric-Based Reinforcement Learning for Open-Ended Reasoning |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2509.25534 |