Reward Models in Deep Reinforcement Learning: A Survey
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911011936665600 |
|---|---|
| author | Yu, Rui Wan, Shenghua Wang, Yucen Gao, Chen-Xiao Gan, Le Zhang, Zongzhang Zhan, De-Chuan |
| author_facet | Yu, Rui Wan, Shenghua Wang, Yucen Gao, Chen-Xiao Gan, Le Zhang, Zongzhang Zhan, De-Chuan |
| contents | In reinforcement learning (RL), agents continually interact with the environment and use the feedback to refine their behavior. To guide policy optimization, reward models are introduced as proxies of the desired objectives, such that when the agent maximizes the accumulated reward, it also fulfills the task designer's intentions. Recently, significant attention from both academic and industrial researchers has focused on developing reward models that not only align closely with the true objectives but also facilitate policy optimization. In this survey, we provide a comprehensive review of reward modeling techniques within the deep RL literature. We begin by outlining the background and preliminaries in reward modeling. Next, we present an overview of recent reward modeling approaches, categorizing them based on the source, the mechanism, and the learning paradigm. Building on this understanding, we discuss various applications of these reward modeling techniques and review methods for evaluating reward models. Finally, we conclude by highlighting promising research directions in reward modeling. Altogether, this survey includes both established and emerging methods, filling the vacancy of a systematic review of reward models in current literature. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_15421 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Reward Models in Deep Reinforcement Learning: A Survey Yu, Rui Wan, Shenghua Wang, Yucen Gao, Chen-Xiao Gan, Le Zhang, Zongzhang Zhan, De-Chuan Machine Learning Artificial Intelligence In reinforcement learning (RL), agents continually interact with the environment and use the feedback to refine their behavior. To guide policy optimization, reward models are introduced as proxies of the desired objectives, such that when the agent maximizes the accumulated reward, it also fulfills the task designer's intentions. Recently, significant attention from both academic and industrial researchers has focused on developing reward models that not only align closely with the true objectives but also facilitate policy optimization. In this survey, we provide a comprehensive review of reward modeling techniques within the deep RL literature. We begin by outlining the background and preliminaries in reward modeling. Next, we present an overview of recent reward modeling approaches, categorizing them based on the source, the mechanism, and the learning paradigm. Building on this understanding, we discuss various applications of these reward modeling techniques and review methods for evaluating reward models. Finally, we conclude by highlighting promising research directions in reward modeling. Altogether, this survey includes both established and emerging methods, filling the vacancy of a systematic review of reward models in current literature. |
| title | Reward Models in Deep Reinforcement Learning: A Survey |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2506.15421 |