Cover Image

Beyond the Binary: Capturing Diverse Preferences With Reward Regularization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Padmakumar, Vishakh, Jin, Chuanyang, Kirk, Hannah Rose, He, He
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Does Writing with Language Models Reduce Content Diversity?
by: Padmakumar, Vishakh, et al.
Published: (2023)

Evaluating the Diversity and Quality of LLM Generated Content
by: Shypula, Alexander, et al.
Published: (2025)

Creativity Support in the Age of Large Language Models: An Empirical Study Involving Emerging Writers
by: Chakrabarty, Tuhin, et al.
Published: (2023)

SparkMe: Adaptive Semi-Structured Interviewing for Qualitative Insight Discovery
by: Anugraha, David, et al.
Published: (2026)

Transformers Struggle to Learn to Search
by: Saparov, Abulhair, et al.
Published: (2024)

Principled Content Selection to Generate Diverse and Personalized Multi-Document Summaries
by: Padmakumar, Vishakh, et al.
Published: (2025)

Multilingual != Multicultural: Evaluating Gaps Between Multilingual Capabilities and Cultural Alignment in LLMs
by: Rystrøm, Jonathan, et al.
Published: (2025)

T-REG: Preference Optimization with Token-Level Reward Regularization
by: Zhou, Wenxuan, et al.
Published: (2024)

No Single Best Model for Diversity: Learning a Router for Sample Diversity
by: Liu, Yuhan, et al.
Published: (2026)

Beyond Overlap Metrics: Rewarding Reasoning and Preferences for Faithful Multi-Role Dialogue Summarization
by: Mei, Xiaoyong, et al.
Published: (2026)

Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty
by: Damani, Mehul, et al.
Published: (2025)

Reward Models Inherit Value Biases from Pretraining
by: Christian, Brian, et al.
Published: (2026)

Can LLMs Capture Human Preferences?
by: Goli, Ali, et al.
Published: (2023)

Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards
by: Wang, Haoxiang, et al.
Published: (2024)

Measuring LLM Novelty As The Frontier Of Original And High-Quality Output
by: Padmakumar, Vishakh, et al.
Published: (2025)

Capturing Bias Diversity in LLMs
by: Gosavi, Purva Prasad, et al.
Published: (2024)

Capturing Nuanced Preferences: Preference-Aligned Distillation for Small Language Models
by: Gu, Yanggan, et al.
Published: (2025)

Larger or Smaller Reward Margins to Select Preferences for Alignment?
by: Huang, Kexin, et al.
Published: (2025)

Spontaneous Reward Hacking in Iterative Self-Refinement
by: Pan, Jane, et al.
Published: (2024)

AlphaDPO: Adaptive Reward Margin for Direct Preference Optimization
by: Wu, Junkang, et al.
Published: (2024)

Beyond Scalar Reward Model: Learning Generative Judge from Preference Data
by: Ye, Ziyi, et al.
Published: (2024)

XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models
by: Röttger, Paul, et al.
Published: (2023)

Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains
by: Gunjal, Anisha, et al.
Published: (2025)

Iterative Reasoning Preference Optimization
by: Pang, Richard Yuanzhe, et al.
Published: (2024)

RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignment
by: Jin, Zhuoran, et al.
Published: (2024)

Reward Model Overoptimisation in Iterated RLHF
by: Wolf, Lorenz, et al.
Published: (2025)

Difficulty-Based Preference Data Selection by DPO Implicit Reward Gap
by: Qi, Xuan, et al.
Published: (2025)

Agentic Reward Modeling: Integrating Human Preferences with Verifiable Correctness Signals for Reliable Reward Systems
by: Peng, Hao, et al.
Published: (2025)

Is It Thinking or Cheating? Detecting Implicit Reward Hacking by Measuring Reasoning Effort
by: Wang, Xinpeng, et al.
Published: (2025)

Misaligned by Reward: Socially Undesirable Preferences in LLMs
by: Ghazaryan, Gayane, et al.
Published: (2026)

AutoToM: Scaling Model-based Mental Inference via Automated Agent Modeling
by: Zhang, Zhining, et al.
Published: (2025)

Towards Understanding the Influence of Reward Margin on Preference Model Performance
by: Qin, Bowen, et al.
Published: (2024)

Implicit Cross-Lingual Rewarding for Efficient Multilingual Preference Alignment
by: Yang, Wen, et al.
Published: (2025)

Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with Free-Form Preferences
by: Jin, Zhuoran, et al.
Published: (2025)

Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models
by: Röttger, Paul, et al.
Published: (2024)

Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs
by: Liu, Chris Yuhao, et al.
Published: (2024)

Regularizing Hidden States Enables Learning Generalizable Reward Model for LLMs
by: Yang, Rui, et al.
Published: (2024)

Beyond Under-Alignment: Atomic Preference Enhanced Factuality Tuning for Large Language Models
by: Yuan, Hongbang, et al.
Published: (2024)

DESTEIN: Navigating Detoxification of Language Models via Universal Steering Pairs and Head-wise Activation Fusion
by: Li, Yu, et al.
Published: (2024)

RMGAP: Benchmarking the Generalization of Reward Models across Diverse Preferences
by: Zhou, Yangyang, et al.
Published: (2026)