What is the objective of reasoning with reinforcement learning?
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911212002869248 |
|---|---|
| author | Davis, Damek Recht, Benjamin |
| author_facet | Davis, Damek Recht, Benjamin |
| contents | We show that several popular algorithms for reinforcement learning in large language models with binary rewards can be viewed as stochastic gradient ascent on a monotone transform of the probability of a correct answer given a prompt. In particular, the transformation associated with rejection sampling algorithms is the logarithm and that associated with the GRPO algorithm is the arcsine of the square root. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_13651 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | What is the objective of reasoning with reinforcement learning? Davis, Damek Recht, Benjamin Machine Learning Optimization and Control We show that several popular algorithms for reinforcement learning in large language models with binary rewards can be viewed as stochastic gradient ascent on a monotone transform of the probability of a correct answer given a prompt. In particular, the transformation associated with rejection sampling algorithms is the logarithm and that associated with the GRPO algorithm is the arcsine of the square root. |
| title | What is the objective of reasoning with reinforcement learning? |
| topic | Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2510.13651 |