Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2505.02666 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915473858232320 |
|---|---|
| author | Ji, Miaomiao Wu, Yanqiu Wu, Zhibin Wang, Shoujin Yang, Jian Dras, Mark Naseem, Usman |
| author_facet | Ji, Miaomiao Wu, Yanqiu Wu, Zhibin Wang, Shoujin Yang, Jian Dras, Mark Naseem, Usman |
| contents | Reward design plays a pivotal role in aligning large language models (LLMs) with human values, serving as the bridge between feedback signals and model optimization. This survey provides a structured organization of reward modeling and addresses three key aspects: mathematical formulation, construction practices, and interaction with optimization paradigms. Building on this, it develops a macro-level taxonomy that characterizes reward mechanisms along complementary dimensions, thereby offering both conceptual clarity and practical guidance for alignment research. The progression of LLM alignment can be understood as a continuous refinement of reward design strategies, with recent developments highlighting paradigm shifts from reinforcement learning (RL)-based to RL-free optimization and from single-task to multi-objective and complex settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_02666 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Survey on Progress in LLM Alignment from the Perspective of Reward Design Ji, Miaomiao Wu, Yanqiu Wu, Zhibin Wang, Shoujin Yang, Jian Dras, Mark Naseem, Usman Computation and Language Reward design plays a pivotal role in aligning large language models (LLMs) with human values, serving as the bridge between feedback signals and model optimization. This survey provides a structured organization of reward modeling and addresses three key aspects: mathematical formulation, construction practices, and interaction with optimization paradigms. Building on this, it develops a macro-level taxonomy that characterizes reward mechanisms along complementary dimensions, thereby offering both conceptual clarity and practical guidance for alignment research. The progression of LLM alignment can be understood as a continuous refinement of reward design strategies, with recent developments highlighting paradigm shifts from reinforcement learning (RL)-based to RL-free optimization and from single-task to multi-objective and complex settings. |
| title | A Survey on Progress in LLM Alignment from the Perspective of Reward Design |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2505.02666 |