Cover Image

VRM: Teaching Reward Models to Understand Authentic Human Preferences

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Biao, Xu, Ning, Yang, Junming, Xu, Hao, Geng, Xin
Format:	Preprint
Published:	2026
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Preference Orchestrator: Prompt-Aware Multi-Objective Alignment for Large Language Models
by: Liu, Biao, et al.
Published: (2025)

Alignment through Meta-Weighted Online Sampling: Bridging the Gap between Data Generation and Preference Optimization
by: Yang, Junming, et al.
Published: (2025)

RoVRM: A Robust Visual Reward Model Optimized via Auxiliary Textual Preference Data
by: Wang, Chenglong, et al.
Published: (2024)

Progressively Label Enhancement for Large Language Model Alignment
by: Liu, Biao, et al.
Published: (2024)

Teaching LLMs to Abstain via Fine-Grained Semantic Confidence Reward
by: An, Hao, et al.
Published: (2025)

Negative-Prompt-driven Alignment for Generative Language Model
by: Qiao, Shiqi, et al.
Published: (2024)

Agentic Reward Modeling: Integrating Human Preferences with Verifiable Correctness Signals for Reliable Reward Systems
by: Peng, Hao, et al.
Published: (2025)

Discriminative Finetuning of Generative Large Language Models without Reward Models and Human Preference Data
by: Guo, Siqi, et al.
Published: (2025)

Towards Understanding the Influence of Reward Margin on Preference Model Performance
by: Qin, Bowen, et al.
Published: (2024)

Teaching Your Models to Understand Code via Focal Preference Alignment
by: Wu, Jie, et al.
Published: (2025)

Enhancing Large Language Model Reasoning with Reward Models: An Analytical Survey
by: Liu, Qiyuan, et al.
Published: (2025)

Boosting Reward Model with Preference-Conditional Multi-Aspect Synthetic Data Generation
by: Shen, Jiaming, et al.
Published: (2024)

Understanding Performance Collapse in Layer-Pruned Large Language Models via Decision Representation Transitions
by: Shi, Boyu, et al.
Published: (2026)

Remedy: Learning Machine Translation Evaluation from Human Preferences with Reward Modeling
by: Tan, Shaomu, et al.
Published: (2025)

XPERT: Expert Knowledge Transfer for Effective Training of Language Models
by: Liu, Chang, et al.
Published: (2026)

Preference Poisoning Attacks on Reward Model Learning
by: Wu, Junlin, et al.
Published: (2024)

UserLM-R1: Modeling Human Reasoning in User Language Models with Multi-Reward Reinforcement Learning
by: Zhang, Feng, et al.
Published: (2026)

Beyond Single-Reward: Multi-Pair, Multi-Perspective Preference Optimization for Machine Translation
by: Wang, Hao, et al.
Published: (2025)

Weighted-Reward Preference Optimization for Implicit Model Fusion
by: Yang, Ziyi, et al.
Published: (2024)

AlignSurvey: A Comprehensive Benchmark for Human Preferences Alignment in Social Surveys
by: Lin, Chenxi, et al.
Published: (2025)

Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergy
by: Liu, Chris Yuhao, et al.
Published: (2025)

Beyond "I cannot fulfill this request": Alleviating Rigid Rejection in LLMs via Label Enhancement
by: Zhang, Ying, et al.
Published: (2026)

Pragmatic Feature Preferences: Learning Reward-Relevant Preferences from Human Input
by: Peng, Andi, et al.
Published: (2024)

Sentence-level Reward Model can Generalize Better for Aligning LLM from Human Preference
by: Qiu, Wenjie, et al.
Published: (2025)

Full-Step-DPO: Self-Supervised Preference Optimization with Step-wise Rewards for Mathematical Reasoning
by: Xu, Huimin, et al.
Published: (2025)

BATON: Aligning Text-to-Audio Model with Human Preference Feedback
by: Liao, Huan, et al.
Published: (2024)

DreamReward: Text-to-3D Generation with Human Preference
by: Ye, Junliang, et al.
Published: (2024)

EVALUESTEER: Measuring Reward Model Steerability Towards Values and Preferences
by: Ghate, Kshitish, et al.
Published: (2025)

Omni-RRM: Advancing Omni Reward Modeling via Automatic Rubric-Grounded Preference Synthesis
by: Kong, Zicheng, et al.
Published: (2026)

Redefining <Creative> in Dictionary: Towards an Enhanced Semantic Understanding of Creative Generation
by: Feng, Fu, et al.
Published: (2024)

CAMEL: Confidence-Gated Reflection for Reward Modeling
by: Zhu, Zirui, et al.
Published: (2026)

BPO: Revisiting Preference Modeling in Direct Preference Optimization
by: Sun, Lin, et al.
Published: (2025)

Probing Preference Representations: A Multi-Dimensional Evaluation and Analysis Method for Reward Models
by: Wang, Chenglong, et al.
Published: (2025)

WildReward: Learning Reward Models from In-the-Wild Human Interactions
by: Peng, Hao, et al.
Published: (2026)

Learning Ordinal Probabilistic Reward from Preferences
by: Chen, Longze, et al.
Published: (2026)

Weak-to-Strong Preference Optimization: Stealing Reward from Weak Aligned Model
by: Zhu, Wenhong, et al.
Published: (2024)

Long-form RewardBench: Evaluating Reward Models for Long-form Generation
by: Huang, Hui, et al.
Published: (2026)

Uncovering Factor Level Preferences to Improve Human-Model Alignment
by: Oh, Juhyun, et al.
Published: (2024)

MANBench: Is Your Multimodal Model Smarter than Human?
by: Zhou, Han, et al.
Published: (2025)

WorldPM: Scaling Human Preference Modeling
by: Wang, Binghai, et al.
Published: (2025)