A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Future
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908323216883712 |
|---|---|
| author | Zhong, Jialun Shen, Wei Li, Yanzeng Gao, Songyang Lu, Hua Chen, Yicheng Zhang, Yang Zhou, Wei Gu, Jinjie Zou, Lei |
| author_facet | Zhong, Jialun Shen, Wei Li, Yanzeng Gao, Songyang Lu, Hua Chen, Yicheng Zhang, Yang Zhou, Wei Gu, Jinjie Zou, Lei |
| contents | Reward Model (RM) has demonstrated impressive potential for enhancing Large Language Models (LLM), as RM can serve as a proxy for human preferences, providing signals to guide LLMs' behavior in various tasks. In this paper, we provide a comprehensive overview of relevant research, exploring RMs from the perspectives of preference collection, reward modeling, and usage. Next, we introduce the applications of RMs and discuss the benchmarks for evaluation. Furthermore, we conduct an in-depth analysis of the challenges existing in the field and dive into the potential research directions. This paper is dedicated to providing beginners with a comprehensive introduction to RMs and facilitating future studies. The resources are publicly available at github\footnote{https://github.com/JLZhong23/awesome-reward-models}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_12328 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Future Zhong, Jialun Shen, Wei Li, Yanzeng Gao, Songyang Lu, Hua Chen, Yicheng Zhang, Yang Zhou, Wei Gu, Jinjie Zou, Lei Computation and Language Artificial Intelligence Reward Model (RM) has demonstrated impressive potential for enhancing Large Language Models (LLM), as RM can serve as a proxy for human preferences, providing signals to guide LLMs' behavior in various tasks. In this paper, we provide a comprehensive overview of relevant research, exploring RMs from the perspectives of preference collection, reward modeling, and usage. Next, we introduce the applications of RMs and discuss the benchmarks for evaluation. Furthermore, we conduct an in-depth analysis of the challenges existing in the field and dive into the potential research directions. This paper is dedicated to providing beginners with a comprehensive introduction to RMs and facilitating future studies. The resources are publicly available at github\footnote{https://github.com/JLZhong23/awesome-reward-models}. |
| title | A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Future |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2504.12328 |