Copertina

When Errors Can Be Beneficial: A Categorization of Imperfect Rewards for Policy Gradient

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Shang, Shuning, Strauss, Hubert, Wei, Stanley, Arora, Sanjeev, Razin, Noam
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

What Makes a Reward Model a Good Teacher? An Optimization Perspective
di: Razin, Noam, et al.
Pubblicazione: (2025)

Why is Your Language Model a Poor Implicit Reward Model?
di: Razin, Noam, et al.
Pubblicazione: (2025)

Understanding Deep Learning via Notions of Rank
di: Razin, Noam
Pubblicazione: (2024)

Lecture Notes on Linear Neural Networks: A Tale of Optimization and Generalization in Deep Learning
di: Cohen, Nadav, et al.
Pubblicazione: (2024)

Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization
di: Razin, Noam, et al.
Pubblicazione: (2024)

Can RLHF be More Efficient with Imperfect Reward Models? A Policy Coverage Perspective
di: Huang, Jiawei, et al.
Pubblicazione: (2025)

Implicit Bias of Policy Gradient in Linear Quadratic Control: Extrapolation to Unseen Initial States
di: Razin, Noam, et al.
Pubblicazione: (2024)

Provable unlearning in topic modeling and downstream tasks
di: Wei, Stanley, et al.
Pubblicazione: (2024)

Vanishing Gradients in Reinforcement Finetuning of Language Models
di: Razin, Noam, et al.
Pubblicazione: (2023)

Contextual Drag: How Errors in the Context Affect LLM Reasoning
di: Cheng, Yun, et al.
Pubblicazione: (2026)

What Makes Data Suitable for a Locally Connected Neural Network? A Necessary and Sufficient Condition Based on Quantum Entanglement
di: Alexander, Yotam, et al.
Pubblicazione: (2023)

Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers
di: Cai, Xin-Qiang, et al.
Pubblicazione: (2025)

An Imperfect Verifier is Good Enough: Learning with Noisy Rewards
di: Plesner, Andreas, et al.
Pubblicazione: (2026)

Hierarchical Apprenticeship Learning from Imperfect Demonstrations with Evolving Rewards
di: Islam, Md Mirajul, et al.
Pubblicazione: (2026)

Can Models Learn Skill Composition from Examples?
di: Zhao, Haoyu, et al.
Pubblicazione: (2024)

What Can You Do When You Have Zero Rewards During RL?
di: Prakash, Jatin, et al.
Pubblicazione: (2025)

Advantage Shaping as Surrogate Reward Maximization: Unifying Pass@K Policy Gradients
di: Thrampoulidis, Christos, et al.
Pubblicazione: (2025)

Stabilizing Policy Gradient Methods via Reward Profiling
di: Ahmed, Shihab, et al.
Pubblicazione: (2025)

Categorical Policies: Multimodal Policy Learning and Exploration in Continuous Control
di: Islam, SM Mazharul, et al.
Pubblicazione: (2025)

When is Off-Policy Evaluation (Reward Modeling) Useful in Contextual Bandits? A Data-Centric Perspective
di: Sun, Hao, et al.
Pubblicazione: (2023)

Skill-Targeted Adaptive Training
di: He, Yinghui, et al.
Pubblicazione: (2025)

Metacognitive Reuse: Turning Recurring LLM Reasoning Into Concise Behaviors
di: Didolkar, Aniket, et al.
Pubblicazione: (2025)

A Policy-Gradient Approach to Solving Imperfect-Information Games with Best-Iterate Convergence
di: Liu, Mingyang, et al.
Pubblicazione: (2024)

Regret Analysis of Policy Gradient Algorithm for Infinite Horizon Average Reward Markov Decision Processes
di: Bai, Qinbo, et al.
Pubblicazione: (2023)

Binary Split Categorical feature with Mean Absolute Error Criteria in CART
di: Yu, Peng, et al.
Pubblicazione: (2025)

Learning General Parameterized Policies for Infinite Horizon Average Reward Constrained MDPs via Primal-Dual Policy Gradient Algorithm
di: Bai, Qinbo, et al.
Pubblicazione: (2024)

Second-Order Actor-Critic Methods for Discounted MDPs via Policy Hessian Decomposition
di: Manivannan, Sanjeev, et al.
Pubblicazione: (2026)

Intrinsic Reward Policy Optimization for Sparse-Reward Environments
di: Cho, Minjae, et al.
Pubblicazione: (2026)

Policy Filtration for RLHF to Mitigate Noise in Reward Models
di: Zhang, Chuheng, et al.
Pubblicazione: (2024)

ORSO: Accelerating Reward Design via Online Reward Selection and Policy Optimization
di: Zhang, Chen Bo Calvin, et al.
Pubblicazione: (2024)

When LLM Reward Design Fails: Diagnostic-Driven Refinement for Sparse Structured RL
di: Wang, Youting, et al.
Pubblicazione: (2026)

Reward Learning through Ranking Mean Squared Error
di: Kharyal, Chaitanya, et al.
Pubblicazione: (2026)

Preference-Based Gradient Estimation for ML-Guided Approximate Combinatorial Optimization
di: Mielke, Arman, et al.
Pubblicazione: (2025)

MAPLE: Multi-State Aggregated Policy Evaluation for AlphaZero in Imperfect-Information Games
di: Li, Qian-Rong, et al.
Pubblicazione: (2026)

Optimal Labeler Assignment and Sampling for Active Learning in the Presence of Imperfect Labels
di: Ahadi, Pouya, et al.
Pubblicazione: (2025)

Sustained Gradient Alignment Mediates Subliminal Learning in a Multi-Step Setting: Evidence from MNIST Auxiliary Logit Distillation Experiment
di: Kitkana, Chayanon, et al.
Pubblicazione: (2026)

Robot Policy Learning with Temporal Optimal Transport Reward
di: Fu, Yuwei, et al.
Pubblicazione: (2024)

Learning General Policies with Policy Gradient Methods
di: Ståhlberg, Simon, et al.
Pubblicazione: (2025)

Unrealized Expectations: Comparing AI Methods vs Classical Algorithms for Maximum Independent Set
di: Wu, Yikai, et al.
Pubblicazione: (2025)

GOPO: Policy Optimization using Ranked Rewards
di: Choi, Kyuseong, et al.
Pubblicazione: (2026)