Cover Image

Reward Models Are Secretly Value Functions: Temporally Coherent Reward Modeling

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Nikulkov, Alex
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

GRPO is Secretly a Process Reward Model
by: Sullivan, Michael, et al.
Published: (2025)

Quasimetric Value Functions with Dense Rewards
by: Valieva, Khadichabonu, et al.
Published: (2024)

RewardBench: Evaluating Reward Models for Language Modeling
by: Lambert, Nathan, et al.
Published: (2024)

Intentionally-underestimated Value Function at Terminal State for Temporal-difference Learning with Mis-designed Reward
by: Kobayashi, Taisuke
Published: (2023)

Coherent Off-Policy Improvement of Large Behavior Models with Learned Rewards
by: Scherer, Christian, et al.
Published: (2026)

Direct Preference Optimization: Your Language Model is Secretly a Reward Model
by: Rafailov, Rafael, et al.
Published: (2023)

Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking
by: Eisenstein, Jacob, et al.
Published: (2023)

Parent-Guided Semantic Reward Model (PGSRM): Embedding-Based Reward Functions for Reinforcement Learning of Transformer Language Models
by: Plashchinsky, Alexandr
Published: (2025)

Generative Reward Models
by: Mahan, Dakota, et al.
Published: (2024)

Uncertainty-aware Reward Model: Teaching Reward Models to Know What is Unknown
by: Lou, Xingzhou, et al.
Published: (2024)

The Value of Reward Lookahead in Reinforcement Learning
by: Merlis, Nadav, et al.
Published: (2024)

FunPRM: Function-as-Step Process Reward Model with Meta Reward Correction for Code Generation
by: Zhang, Ruiyi, et al.
Published: (2026)

Inference-Time Reward Hacking in Large Language Models
by: Khalaf, Hadi, et al.
Published: (2025)

Reward Under Attack: Analyzing the Robustness and Hackability of Process Reward Models
by: Tiwari, Rishabh, et al.
Published: (2026)

Rewarding DINO: Predicting Dense Rewards with Vision Foundation Models
by: Krack, Pierre, et al.
Published: (2026)

Repairing Reward Functions with Feedback to Mitigate Reward Hacking
by: Hatgis-Kessell, Stephane, et al.
Published: (2025)

Unsupervised Process Reward Models
by: Gadetsky, Artyom, et al.
Published: (2026)

Probabilistic Uncertain Reward Model
by: Sun, Wangtao, et al.
Published: (2025)

Adversarial Training of Reward Models
by: Bukharin, Alexander, et al.
Published: (2025)

Information-Theoretic Reward Modeling for Stable RLHF: Detecting and Mitigating Reward Hacking
by: Miao, Yuchun, et al.
Published: (2025)

Reward Models Inherit Value Biases from Pretraining
by: Christian, Brian, et al.
Published: (2026)

Refining Multidimensional Video Reward Models via Disentangled Influence Functions
by: Wang, Muyao, et al.
Published: (2026)

TDRM: Smooth Reward Models with Temporal Difference for LLM RL and Inference
by: Zhang, Dan, et al.
Published: (2025)

Pretrain Value, Not Reward: Decoupled Value Policy Optimization
by: Huang, Chenghua, et al.
Published: (2025)

RewardAnything: Generalizable Principle-Following Reward Models
by: Yu, Zhuohao, et al.
Published: (2025)

Beyond Simple Sum of Delayed Rewards: Non-Markovian Reward Modeling for Reinforcement Learning
by: Tang, Yuting, et al.
Published: (2024)

Personalized RewardBench: Evaluating Reward Models with Human Aligned Personalization
by: Ma, Qiyao, et al.
Published: (2026)

Critique-out-Loud Reward Models
by: Ankner, Zachary, et al.
Published: (2024)

Optimal Design for Reward Modeling in RLHF
by: Scheid, Antoine, et al.
Published: (2024)

Bayesian Reward Models for LLM Alignment
by: Yang, Adam X., et al.
Published: (2024)

SemiReward: A General Reward Model for Semi-supervised Learning
by: Li, Siyuan, et al.
Published: (2023)

Beyond Reward Hacking: Causal Rewards for Large Language Model Alignment
by: Wang, Chaoqi, et al.
Published: (2025)

Constraints as Rewards: Reinforcement Learning for Robots without Reward Functions
by: Ishihara, Yu, et al.
Published: (2025)

Informativeness of Reward Functions in Reinforcement Learning
by: Devidze, Rati, et al.
Published: (2024)

SoliReward: Mitigating Susceptibility to Reward Hacking and Annotation Noise in Video Generation Reward Models
by: Lian, Jiesong, et al.
Published: (2025)

Implicit Updates for Average-Reward Temporal Difference Learning
by: Kim, Hwanwoo, et al.
Published: (2025)

M-RewardBench: Evaluating Reward Models in Multilingual Settings
by: Gureja, Srishti, et al.
Published: (2024)

Rethinking Reward Model Evaluation Through the Lens of Reward Overoptimization
by: Kim, Sunghwan, et al.
Published: (2025)

What Fundamental Structure in Reward Functions Enables Efficient Sparse-Reward Learning?
by: Shihab, Ibne Farabi, et al.
Published: (2025)

Improving Generative Ad Text on Facebook using Reinforcement Learning
by: Jiang, Daniel R., et al.
Published: (2025)