Reinforcing Chain-of-Thought Reasoning with Self-Evolving Rubrics
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910019046342656 |
|---|---|
| author | Sheng, Leheng Ma, Wenchang Hong, Ruixin Wang, Xiang Zhang, An Chua, Tat-Seng |
| author_facet | Sheng, Leheng Ma, Wenchang Hong, Ruixin Wang, Xiang Zhang, An Chua, Tat-Seng |
| contents | Despite chain-of-thought (CoT) playing crucial roles in LLM reasoning, directly rewarding it is difficult: training a reward model demands heavy human labeling efforts, and static RMs struggle with evolving CoT distributions and reward hacking. These challenges motivate us to seek an autonomous CoT rewarding approach that requires no human annotation efforts and can evolve gradually. Inspired by recent self-evolving training methods, we propose \textbf{RLCER} (\textbf{R}einforcement \textbf{L}earning with \textbf{C}oT Supervision via Self-\textbf{E}volving \textbf{R}ubrics), which enhances the outcome-centric RLVR by rewarding CoTs with self-proposed and self-evolving rubrics. We show that self-proposed and self-evolving rubrics provide reliable CoT supervision signals even without outcome rewards, enabling RLCER to outperform outcome-centric RLVR. Moreover, when used as in-prompt hints, these self-proposed rubrics further improve inference-time performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_10885 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Reinforcing Chain-of-Thought Reasoning with Self-Evolving Rubrics Sheng, Leheng Ma, Wenchang Hong, Ruixin Wang, Xiang Zhang, An Chua, Tat-Seng Artificial Intelligence Machine Learning Despite chain-of-thought (CoT) playing crucial roles in LLM reasoning, directly rewarding it is difficult: training a reward model demands heavy human labeling efforts, and static RMs struggle with evolving CoT distributions and reward hacking. These challenges motivate us to seek an autonomous CoT rewarding approach that requires no human annotation efforts and can evolve gradually. Inspired by recent self-evolving training methods, we propose \textbf{RLCER} (\textbf{R}einforcement \textbf{L}earning with \textbf{C}oT Supervision via Self-\textbf{E}volving \textbf{R}ubrics), which enhances the outcome-centric RLVR by rewarding CoTs with self-proposed and self-evolving rubrics. We show that self-proposed and self-evolving rubrics provide reliable CoT supervision signals even without outcome rewards, enabling RLCER to outperform outcome-centric RLVR. Moreover, when used as in-prompt hints, these self-proposed rubrics further improve inference-time performance. |
| title | Reinforcing Chain-of-Thought Reasoning with Self-Evolving Rubrics |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2602.10885 |