Copertina

Why is Your Language Model a Poor Implicit Reward Model?

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Razin, Noam, Lin, Yong, Yao, Jiarui, Arora, Sanjeev
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language Artificial Intelligence Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

What Makes a Reward Model a Good Teacher? An Optimization Perspective
di: Razin, Noam, et al.
Pubblicazione: (2025)

Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization
di: Razin, Noam, et al.
Pubblicazione: (2024)

When Errors Can Be Beneficial: A Categorization of Imperfect Rewards for Policy Gradient
di: Shang, Shuning, et al.
Pubblicazione: (2026)

Vanishing Gradients in Reinforcement Finetuning of Language Models
di: Razin, Noam, et al.
Pubblicazione: (2023)

Direct Preference Optimization: Your Language Model is Secretly a Reward Model
di: Rafailov, Rafael, et al.
Pubblicazione: (2023)

Understanding Deep Learning via Notions of Rank
di: Razin, Noam
Pubblicazione: (2024)

Process Reinforcement through Implicit Rewards
di: Cui, Ganqu, et al.
Pubblicazione: (2025)

DavIR: Data Selection via Implicit Reward for Large Language Models
di: Zhou, Haotian, et al.
Pubblicazione: (2023)

Can Models Learn Skill Composition from Examples?
di: Zhao, Haoyu, et al.
Pubblicazione: (2024)

On the Robustness of Reward Models for Language Model Alignment
di: Hong, Jiwoo, et al.
Pubblicazione: (2025)

Ineq-Comp: Benchmarking Human-Intuitive Compositional Reasoning in Automated Theorem Proving on Inequalities
di: Zhao, Haoyu, et al.
Pubblicazione: (2025)

RewardFlow: Topology-Aware Reward Propagation on State Graphs for Agentic RL with Large Language Models
di: Feng, Xiao, et al.
Pubblicazione: (2026)

ReDit: Reward Dithering for Improved LLM Policy Optimization
di: Wei, Chenxing, et al.
Pubblicazione: (2025)

Contextual Drag: How Errors in the Context Affect LLM Reasoning
di: Cheng, Yun, et al.
Pubblicazione: (2026)

Implicit Personalization in Language Models: A Systematic Study
di: Jin, Zhijing, et al.
Pubblicazione: (2024)

Self-Generated Critiques Boost Reward Modeling for Language Models
di: Yu, Yue, et al.
Pubblicazione: (2024)

Fine-tuning Language Models with Generative Adversarial Reward Modelling
di: Yu, Zhang Ze, et al.
Pubblicazione: (2023)

Are Retrials All You Need? Enhancing Large Language Model Reasoning Without Verbalized Feedback
di: Potamitis, Nearchos, et al.
Pubblicazione: (2025)

Towards Large Language Models that Benefit for All: Benchmarking Group Fairness in Reward Models
di: Song, Kefan, et al.
Pubblicazione: (2025)

Lecture Notes on Linear Neural Networks: A Tale of Optimization and Generalization in Deep Learning
di: Cohen, Nadav, et al.
Pubblicazione: (2024)

Self-Play with Adversarial Critic: Provable and Scalable Offline Alignment for Language Models
di: Ji, Xiang, et al.
Pubblicazione: (2024)

Reviving The Classics: Active Reward Modeling in Large Language Model Alignment
di: Shen, Yunyi, et al.
Pubblicazione: (2025)

ALaRM: Align Language Models via Hierarchical Rewards Modeling
di: Lai, Yuhang, et al.
Pubblicazione: (2024)

Language Models Can Learn from Verbal Feedback Without Scalar Rewards
di: Luo, Renjie, et al.
Pubblicazione: (2025)

Fine-Tuning Language Models with Reward Learning on Policy
di: Lang, Hao, et al.
Pubblicazione: (2024)

Rethinking the Role of Proxy Rewards in Language Model Alignment
di: Kim, Sungdong, et al.
Pubblicazione: (2024)

Text2Reward: Reward Shaping with Language Models for Reinforcement Learning
di: Xie, Tianbao, et al.
Pubblicazione: (2023)

Intent Factored Generation: Unleashing the Diversity in Your Language Model
di: Ahmed, Eltayeb, et al.
Pubblicazione: (2025)

LESS: Selecting Influential Data for Targeted Instruction Tuning
di: Xia, Mengzhou, et al.
Pubblicazione: (2024)

How to Evaluate Reward Models for RLHF
di: Frick, Evan, et al.
Pubblicazione: (2024)

The Lessons of Developing Process Reward Models in Mathematical Reasoning
di: Zhang, Zhenru, et al.
Pubblicazione: (2025)

Feedback Loops With Language Models Drive In-Context Reward Hacking
di: Pan, Alexander, et al.
Pubblicazione: (2024)

Entropy Aware Reward Guidance for Diffusion Language Model Alignment
di: Tejaswi, Atula, et al.
Pubblicazione: (2026)

Difficulty-Based Preference Data Selection by DPO Implicit Reward Gap
di: Qi, Xuan, et al.
Pubblicazione: (2025)

Implicit Reward as the Bridge: A Unified View of SFT and DPO Connections
di: Wang, Bo, et al.
Pubblicazione: (2025)

Why Larger Language Models Do In-context Learning Differently?
di: Shi, Zhenmei, et al.
Pubblicazione: (2024)

CoLD: Counterfactually-Guided Length Debiasing for Process Reward Models in Mathematical Reasoning
di: Zheng, Congmin, et al.
Pubblicazione: (2025)

From Faithfulness to Correctness: Generative Reward Models that Think Critically
di: Ma, Qiyao, et al.
Pubblicazione: (2025)

It Takes Two: On the Seamlessness between Reward and Policy Model in RLHF
di: Lu, Taiming, et al.
Pubblicazione: (2024)

RewardAnything: Generalizable Principle-Following Reward Models
di: Yu, Zhuohao, et al.
Pubblicazione: (2025)