Cover Image

One Bias After Another: Mechanistic Reward Shaping and Persistent Biases in Language Reward Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Fein, Daniel, Lamparth, Max, Xiang, Violet, Kochenderfer, Mykel J., Haber, Nick
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Reward Bias Substitution: Single-Axis Bias Mitigations Redirect Optimization Pressure
by: Lamparth, Max, et al.
Published: (2026)

LitBench: A Benchmark and Dataset for Reliable Evaluation of Creative Writing
by: Fein, Daniel, et al.
Published: (2025)

Analyzing And Editing Inner Mechanisms Of Backdoored Language Models
by: Lamparth, Max, et al.
Published: (2023)

Risks from Language Models for Automated Mental Healthcare: Ethics and Structure for Implementation
by: Grabb, Declan, et al.
Published: (2024)

Measuring Free-Form Decision-Making Inconsistency of Language Models in Military Crisis Simulations
by: Shrivastava, Aryan, et al.
Published: (2024)

Text2Reward: Reward Shaping with Language Models for Reinforcement Learning
by: Xie, Tianbao, et al.
Published: (2023)

BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practices
by: Reuel, Anka, et al.
Published: (2024)

Reward Shaping to Mitigate Reward Hacking in RLHF
by: Fu, Jiayi, et al.
Published: (2025)

Detecting Prefix Bias in LLM-based Reward Models
by: Kumar, Ashwin, et al.
Published: (2025)

Hypothetical Minds: Scaffolding Theory of Mind for Multi-Agent Tasks with Large Language Models
by: Cross, Logan, et al.
Published: (2024)

Reward Model Perspectives: Whose Opinions Do Reward Models Reward?
by: Elle
Published: (2025)

Self-Rewarding Language Models
by: Yuan, Weizhe, et al.
Published: (2024)

Reward Models Inherit Value Biases from Pretraining
by: Christian, Brian, et al.
Published: (2026)

Rejected Dialects: Biases Against African American Language in Reward Models
by: Mire, Joel, et al.
Published: (2025)

Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs
by: Liu, Chris Yuhao, et al.
Published: (2024)

MemoryRewardBench: Benchmarking Reward Models for Long-Term Memory Management in Large Language Models
by: Tang, Zecheng, et al.
Published: (2026)

Shaping Explanations: Semantic Reward Modeling with Encoder-Only Transformers for GRPO
by: Pappone, Francesco, et al.
Published: (2025)

ResearchCodeBench: Benchmarking LLMs on Implementing Novel Machine Learning Research Code
by: Hua, Tianyu, et al.
Published: (2025)

Entangled in Representations: Mechanistic Investigation of Cultural Biases in Large Language Models
by: Yu, Haeun, et al.
Published: (2025)

Process-based Self-Rewarding Language Models
by: Zhang, Shimao, et al.
Published: (2025)

One Adapts to Any: Meta Reward Modeling for Personalized LLM Alignment
by: Cai, Hongru, et al.
Published: (2026)

GRAM: A Generative Foundation Reward Model for Reward Generalization
by: Wang, Chenglong, et al.
Published: (2025)

SAFER: Probing Safety in Reward Models with Sparse Autoencoder
by: Shi, Wei, et al.
Published: (2025)

Tiny Reward Models
by: Pan, Sarah
Published: (2025)

WildReward: Learning Reward Models from In-the-Wild Human Interactions
by: Peng, Hao, et al.
Published: (2026)

Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Models
by: Albalak, Alon, et al.
Published: (2025)

Post-hoc Reward Calibration: A Case Study on Length Bias
by: Huang, Zeyu, et al.
Published: (2024)

RewardUQ: A Unified Framework for Uncertainty-Aware Reward Models
by: Yang, Daniel, et al.
Published: (2026)

Evaluating and Improving Cultural Awareness of Reward Models for LLM Alignment
by: Zhang, Hongbin, et al.
Published: (2025)

On the Robustness of Reward Models for Language Model Alignment
by: Hong, Jiwoo, et al.
Published: (2025)

EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing
by: Wu, Keming, et al.
Published: (2025)

Bias Vector: Mitigating Biases in Language Models with Task Arithmetic Approach
by: Shirafuji, Daiki, et al.
Published: (2024)

RewardAnything: Generalizable Principle-Following Reward Models
by: Yu, Zhuohao, et al.
Published: (2025)

GTPO and GRPO-S: Token and Sequence-Level Reward Shaping with Policy Entropy
by: Tan, Hongze, et al.
Published: (2025)

Sequence to Sequence Reward Modeling: Improving RLHF by Language Feedback
by: Zhou, Jiayi, et al.
Published: (2024)

Segmenting Text and Learning Their Rewards for Improved RLHF in Language Model
by: Yin, Yueqin, et al.
Published: (2025)

Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models
by: Denison, Carson, et al.
Published: (2024)

When Distance Distracts: Representation Distance Bias in BT-Loss for Reward Models
by: Xie, Tong, et al.
Published: (2025)

Human vs. Machine: Behavioral Differences Between Expert Humans and Language Models in Wargame Simulations
by: Lamparth, Max, et al.
Published: (2024)

The Past Is Not Past: Memory-Enhanced Dynamic Reward Shaping
by: Liu, Yang, et al.
Published: (2026)