Copertina

What Makes a Reward Model a Good Teacher? An Optimization Perspective

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Razin, Noam, Wang, Zixuan, Strauss, Hubert, Wei, Stanley, Lee, Jason D., Arora, Sanjeev
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Artificial Intelligence Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

When Errors Can Be Beneficial: A Categorization of Imperfect Rewards for Policy Gradient
di: Shang, Shuning, et al.
Pubblicazione: (2026)

Why is Your Language Model a Poor Implicit Reward Model?
di: Razin, Noam, et al.
Pubblicazione: (2025)

Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization
di: Razin, Noam, et al.
Pubblicazione: (2024)

Lecture Notes on Linear Neural Networks: A Tale of Optimization and Generalization in Deep Learning
di: Cohen, Nadav, et al.
Pubblicazione: (2024)

Understanding Deep Learning via Notions of Rank
di: Razin, Noam
Pubblicazione: (2024)

What Makes Data Suitable for a Locally Connected Neural Network? A Necessary and Sufficient Condition Based on Quantum Entanglement
di: Alexander, Yotam, et al.
Pubblicazione: (2023)

Vanishing Gradients in Reinforcement Finetuning of Language Models
di: Razin, Noam, et al.
Pubblicazione: (2023)

What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning
di: Liu, Wei, et al.
Pubblicazione: (2023)

Deconstructing What Makes a Good Optimizer for Language Models
di: Zhao, Rosie, et al.
Pubblicazione: (2024)

Can Models Learn Skill Composition from Examples?
di: Zhao, Haoyu, et al.
Pubblicazione: (2024)

Provable unlearning in topic modeling and downstream tasks
di: Wei, Stanley, et al.
Pubblicazione: (2024)

Reward Collapse in Aligning Large Language Models
di: Song, Ziang, et al.
Pubblicazione: (2023)

Contextual Drag: How Errors in the Context Affect LLM Reasoning
di: Cheng, Yun, et al.
Pubblicazione: (2026)

Direct Preference Optimization: Your Language Model is Secretly a Reward Model
di: Rafailov, Rafael, et al.
Pubblicazione: (2023)

The Power of Power Law: Asymmetry Enables Compositional Reasoning
di: Wang, Zixuan, et al.
Pubblicazione: (2026)

LESS: Selecting Influential Data for Targeted Instruction Tuning
di: Xia, Mengzhou, et al.
Pubblicazione: (2024)

Process Reward Models That Think
di: Khalifa, Muhammad, et al.
Pubblicazione: (2025)

What Makes a Good Diffusion Planner for Decision Making?
di: Lu, Haofei, et al.
Pubblicazione: (2025)

Rethinking Reward Model Evaluation Through the Lens of Reward Overoptimization
di: Kim, Sunghwan, et al.
Pubblicazione: (2025)

Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation
di: Yang, Wenkai, et al.
Pubblicazione: (2026)

Efficient Process Reward Modeling via Contrastive Mutual Information
di: Lee, Nakyung, et al.
Pubblicazione: (2026)

What Makes a Good Response? An Empirical Analysis of Quality in Qualitative Interviews
di: Ivey, Jonathan, et al.
Pubblicazione: (2026)

RewardAnything: Generalizable Principle-Following Reward Models
di: Yu, Zhuohao, et al.
Pubblicazione: (2025)

Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates
di: Lyu, Kaifeng, et al.
Pubblicazione: (2024)

Comparing Bad Apples to Good Oranges: Aligning Large Language Models via Joint Preference Optimization
di: Bansal, Hritik, et al.
Pubblicazione: (2024)

ReDit: Reward Dithering for Improved LLM Policy Optimization
di: Wei, Chenxing, et al.
Pubblicazione: (2025)

AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators
di: Mazumder, Aritra, et al.
Pubblicazione: (2026)

On the Robustness of Reward Models for Language Model Alignment
di: Hong, Jiwoo, et al.
Pubblicazione: (2025)

REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewards
di: Stojanovski, Zafir, et al.
Pubblicazione: (2025)

Response-Level Rewards Are All You Need for Online Reinforcement Learning in LLMs: A Mathematical Perspective
di: He, Shenghua, et al.
Pubblicazione: (2025)

Hindsight-Anchored Policy Optimization: Turning Failure into Feedback in Sparse Reward Settings
di: Wu, Yuning, et al.
Pubblicazione: (2026)

Online Intrinsic Rewards for Decision Making Agents from Large Language Model Feedback
di: Zheng, Qinqing, et al.
Pubblicazione: (2024)

PerPO: Perceptual Preference Optimization via Discriminative Rewarding
di: Zhu, Zining, et al.
Pubblicazione: (2025)

What Level of Automation is "Good Enough"? A Benchmark of Large Language Models for Meta-Analysis Data Extraction
di: Li, Lingbo, et al.
Pubblicazione: (2025)

FutureFill: Fast Generation from Convolutional Sequence Models
di: Agarwal, Naman, et al.
Pubblicazione: (2024)

Evaluating Robustness of Reward Models for Mathematical Reasoning
di: Kim, Sunghwan, et al.
Pubblicazione: (2024)

Automated Optimization Modeling via a Localizable Error-Driven Perspective
di: Liu, Weiting, et al.
Pubblicazione: (2026)

How to Evaluate Reward Models for RLHF
di: Frick, Evan, et al.
Pubblicazione: (2024)

Reward Model Perspectives: Whose Opinions Do Reward Models Reward?
di: Elle
Pubblicazione: (2025)

ALaRM: Align Language Models via Hierarchical Rewards Modeling
di: Lai, Yuhang, et al.
Pubblicazione: (2024)