Image de couverture de livre

What Makes a Reward Model a Good Teacher? An Optimization Perspective

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Razin, Noam, Wang, Zixuan, Strauss, Hubert, Wei, Stanley, Lee, Jason D., Arora, Sanjeev
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

When Errors Can Be Beneficial: A Categorization of Imperfect Rewards for Policy Gradient
par: Shang, Shuning, et autres
Publié: (2026)

Why is Your Language Model a Poor Implicit Reward Model?
par: Razin, Noam, et autres
Publié: (2025)

Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization
par: Razin, Noam, et autres
Publié: (2024)

Lecture Notes on Linear Neural Networks: A Tale of Optimization and Generalization in Deep Learning
par: Cohen, Nadav, et autres
Publié: (2024)

Understanding Deep Learning via Notions of Rank
par: Razin, Noam
Publié: (2024)

What Makes Data Suitable for a Locally Connected Neural Network? A Necessary and Sufficient Condition Based on Quantum Entanglement
par: Alexander, Yotam, et autres
Publié: (2023)

Vanishing Gradients in Reinforcement Finetuning of Language Models
par: Razin, Noam, et autres
Publié: (2023)

What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning
par: Liu, Wei, et autres
Publié: (2023)

Deconstructing What Makes a Good Optimizer for Language Models
par: Zhao, Rosie, et autres
Publié: (2024)

Can Models Learn Skill Composition from Examples?
par: Zhao, Haoyu, et autres
Publié: (2024)

Provable unlearning in topic modeling and downstream tasks
par: Wei, Stanley, et autres
Publié: (2024)

Reward Collapse in Aligning Large Language Models
par: Song, Ziang, et autres
Publié: (2023)

Contextual Drag: How Errors in the Context Affect LLM Reasoning
par: Cheng, Yun, et autres
Publié: (2026)

Direct Preference Optimization: Your Language Model is Secretly a Reward Model
par: Rafailov, Rafael, et autres
Publié: (2023)

The Power of Power Law: Asymmetry Enables Compositional Reasoning
par: Wang, Zixuan, et autres
Publié: (2026)

LESS: Selecting Influential Data for Targeted Instruction Tuning
par: Xia, Mengzhou, et autres
Publié: (2024)

Process Reward Models That Think
par: Khalifa, Muhammad, et autres
Publié: (2025)

What Makes a Good Diffusion Planner for Decision Making?
par: Lu, Haofei, et autres
Publié: (2025)

Rethinking Reward Model Evaluation Through the Lens of Reward Overoptimization
par: Kim, Sunghwan, et autres
Publié: (2025)

Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation
par: Yang, Wenkai, et autres
Publié: (2026)

Efficient Process Reward Modeling via Contrastive Mutual Information
par: Lee, Nakyung, et autres
Publié: (2026)

What Makes a Good Response? An Empirical Analysis of Quality in Qualitative Interviews
par: Ivey, Jonathan, et autres
Publié: (2026)

RewardAnything: Generalizable Principle-Following Reward Models
par: Yu, Zhuohao, et autres
Publié: (2025)

Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates
par: Lyu, Kaifeng, et autres
Publié: (2024)

Comparing Bad Apples to Good Oranges: Aligning Large Language Models via Joint Preference Optimization
par: Bansal, Hritik, et autres
Publié: (2024)

ReDit: Reward Dithering for Improved LLM Policy Optimization
par: Wei, Chenxing, et autres
Publié: (2025)

AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators
par: Mazumder, Aritra, et autres
Publié: (2026)

On the Robustness of Reward Models for Language Model Alignment
par: Hong, Jiwoo, et autres
Publié: (2025)

REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewards
par: Stojanovski, Zafir, et autres
Publié: (2025)

Response-Level Rewards Are All You Need for Online Reinforcement Learning in LLMs: A Mathematical Perspective
par: He, Shenghua, et autres
Publié: (2025)

Hindsight-Anchored Policy Optimization: Turning Failure into Feedback in Sparse Reward Settings
par: Wu, Yuning, et autres
Publié: (2026)

Online Intrinsic Rewards for Decision Making Agents from Large Language Model Feedback
par: Zheng, Qinqing, et autres
Publié: (2024)

PerPO: Perceptual Preference Optimization via Discriminative Rewarding
par: Zhu, Zining, et autres
Publié: (2025)

FutureFill: Fast Generation from Convolutional Sequence Models
par: Agarwal, Naman, et autres
Publié: (2024)

What Level of Automation is "Good Enough"? A Benchmark of Large Language Models for Meta-Analysis Data Extraction
par: Li, Lingbo, et autres
Publié: (2025)

Evaluating Robustness of Reward Models for Mathematical Reasoning
par: Kim, Sunghwan, et autres
Publié: (2024)

Automated Optimization Modeling via a Localizable Error-Driven Perspective
par: Liu, Weiting, et autres
Publié: (2026)

How to Evaluate Reward Models for RLHF
par: Frick, Evan, et autres
Publié: (2024)

Reward Model Perspectives: Whose Opinions Do Reward Models Reward?
par: Elle
Publié: (2025)

ALaRM: Align Language Models via Hierarchical Rewards Modeling
par: Lai, Yuhang, et autres
Publié: (2024)