R3: Robust Rubric-Agnostic Reward Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909798099845120 |
|---|---|
| author | Anugraha, David Tang, Zilu Miranda, Lester James V. Zhao, Hanyang Farhansyah, Mohammad Rifqi Kuwanto, Garry Wijaya, Derry Winata, Genta Indra |
| author_facet | Anugraha, David Tang, Zilu Miranda, Lester James V. Zhao, Hanyang Farhansyah, Mohammad Rifqi Kuwanto, Garry Wijaya, Derry Winata, Genta Indra |
| contents | Reward models are essential for aligning language model outputs with human preferences, yet existing approaches often lack both controllability and interpretability. These models are typically optimized for narrow objectives, limiting their generalizability to broader downstream tasks. Moreover, their scalar outputs are difficult to interpret without contextual reasoning. To address these limitations, we introduce $\shortmethodname$, a novel reward modeling framework that is rubric-agnostic, generalizable across evaluation dimensions, and provides interpretable, reasoned score assignments. $\shortmethodname$ enables more transparent and flexible evaluation of language models, supporting robust alignment with diverse human values and use cases. Our models, data, and code are available as open source at https://github.com/rubricreward/r3. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_13388 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | R3: Robust Rubric-Agnostic Reward Models Anugraha, David Tang, Zilu Miranda, Lester James V. Zhao, Hanyang Farhansyah, Mohammad Rifqi Kuwanto, Garry Wijaya, Derry Winata, Genta Indra Computation and Language Artificial Intelligence Machine Learning Reward models are essential for aligning language model outputs with human preferences, yet existing approaches often lack both controllability and interpretability. These models are typically optimized for narrow objectives, limiting their generalizability to broader downstream tasks. Moreover, their scalar outputs are difficult to interpret without contextual reasoning. To address these limitations, we introduce $\shortmethodname$, a novel reward modeling framework that is rubric-agnostic, generalizable across evaluation dimensions, and provides interpretable, reasoned score assignments. $\shortmethodname$ enables more transparent and flexible evaluation of language models, supporting robust alignment with diverse human values and use cases. Our models, data, and code are available as open source at https://github.com/rubricreward/r3. |
| title | R3: Robust Rubric-Agnostic Reward Models |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2505.13388 |