Learning to Self-Verify Makes Language Models Better Reasoners
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911431271645184 |
|---|---|
| author | Chen, Yuxin Wang, Yu Zhang, Yi Ye, Ziang Cai, Zhengzhou Shi, Yaorui Gu, Qi Su, Hui Cai, Xunliang Wang, Xiang Zhang, An Chua, Tat-Seng |
| author_facet | Chen, Yuxin Wang, Yu Zhang, Yi Ye, Ziang Cai, Zhengzhou Shi, Yaorui Gu, Qi Su, Hui Cai, Xunliang Wang, Xiang Zhang, An Chua, Tat-Seng |
| contents | Recent large language models (LLMs) achieve strong performance in generating promising reasoning paths for complex tasks. However, despite powerful generation ability, LLMs remain weak at verifying their own answers, revealing a persistent capability asymmetry between generation and self-verification. In this work, we conduct an in-depth investigation of this asymmetry throughout training evolution and show that, even on the same task, improving generation does not lead to corresponding improvements in self-verification. Interestingly, we find that the reverse direction of this asymmetry behaves differently: learning to self-verify can effectively improve generation performance, achieving accuracy comparable to standard generation training while yielding more efficient and effective reasoning traces. Building on this observation, we further explore integrating self-verification into generation training by formulating a multi-task reinforcement learning framework, where generation and self-verification are optimized as two independent but complementary objectives. Extensive experiments across benchmarks and models demonstrate performance gains over generation-only training in both generation and verification capabilities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_07594 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Learning to Self-Verify Makes Language Models Better Reasoners Chen, Yuxin Wang, Yu Zhang, Yi Ye, Ziang Cai, Zhengzhou Shi, Yaorui Gu, Qi Su, Hui Cai, Xunliang Wang, Xiang Zhang, An Chua, Tat-Seng Computation and Language Artificial Intelligence Recent large language models (LLMs) achieve strong performance in generating promising reasoning paths for complex tasks. However, despite powerful generation ability, LLMs remain weak at verifying their own answers, revealing a persistent capability asymmetry between generation and self-verification. In this work, we conduct an in-depth investigation of this asymmetry throughout training evolution and show that, even on the same task, improving generation does not lead to corresponding improvements in self-verification. Interestingly, we find that the reverse direction of this asymmetry behaves differently: learning to self-verify can effectively improve generation performance, achieving accuracy comparable to standard generation training while yielding more efficient and effective reasoning traces. Building on this observation, we further explore integrating self-verification into generation training by formulating a multi-task reinforcement learning framework, where generation and self-verification are optimized as two independent but complementary objectives. Extensive experiments across benchmarks and models demonstrate performance gains over generation-only training in both generation and verification capabilities. |
| title | Learning to Self-Verify Makes Language Models Better Reasoners |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2602.07594 |