Cover Image

The Reward Model Selection Crisis in Personalized Alignment

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Rezk, Fady, Pan, Yuangang, Foo, Chuan-Sheng, Xu, Xun, Chen, Nancy, Gouk, Henry, Hospedales, Timothy
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Is Scaling Learned Optimizers Worth It? Evaluating The Value of VeLO's 4000 TPU Months
by: Rezk, Fady, et al.
Published: (2023)

Test-Time Alignment of Text-to-Image Diffusion Models via Null-Text Embedding Optimisation
by: Kim, Taehoon, et al.
Published: (2025)

Model Diffusion for Certifiable Few-shot Transfer Learning
by: Rezk, Fady, et al.
Published: (2025)

Model Merging is Secretly Certifiable: Non-Vacuous Generalisation Bounds for Low-Shot Learning
by: Kim, Taehoon, et al.
Published: (2025)

FedP$^2$EFT: Federated Learning to Personalize PEFT for Multilingual LLMs
by: Lee, Royson, et al.
Published: (2025)

On the Limitations of General Purpose Domain Generalisation Methods
by: Gouk, Henry, et al.
Published: (2022)

Better Later Than Sooner: Neuro-Symbolic Knowledge Graph Construction via Ontology-grounded Post-extraction Correction
by: Loconte, Lorenzo, et al.
Published: (2026)

FW-Merging: Scaling Model Merging with Frank-Wolfe Optimization
by: Chen, Hao Mark, et al.
Published: (2025)

Beyond Reward Hacking: Causal Rewards for Large Language Model Alignment
by: Wang, Chaoqi, et al.
Published: (2025)

ConceptPrune: Concept Editing in Diffusion Models via Skilled Neuron Pruning
by: Chavhan, Ruchika, et al.
Published: (2024)

MuPHI: Learning Implicit Multimodal Harm Reasoning via Semantically Grounded Reward Optimization
by: Saha, Anisha, et al.
Published: (2026)

Helpful or Harmful Data? Fine-tuning-free Shapley Attribution for Explaining Language Model Predictions
by: Wang, Jingtan, et al.
Published: (2024)

Self-Supervised Multimodal Learning: A Survey
by: Zong, Yongshuo, et al.
Published: (2023)

DUET: Optimizing Training Data Mixtures via Feedback from Unseen Evaluation Tasks
by: Chen, Zhiliang, et al.
Published: (2025)

Larger or Smaller Reward Margins to Select Preferences for Alignment?
by: Huang, Kexin, et al.
Published: (2025)

PARM: Multi-Objective Test-Time Alignment via Preference-Aware Autoregressive Reward Model
by: Lin, Baijiong, et al.
Published: (2025)

The Chicken and Egg Dilemma: Co-optimizing Data and Model Configurations for LLMs
by: Chen, Zhiliang, et al.
Published: (2026)

A Sobering Look at Tabular Data Generation via Probabilistic Circuits
by: Scassola, Davide, et al.
Published: (2026)

Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment
by: Yang, Rui, et al.
Published: (2024)

Source Attribution for Large Language Model-Generated Data
by: Wang, Jingtan, et al.
Published: (2023)

SALMON: Self-Alignment with Instructable Reward Models
by: Sun, Zhiqing, et al.
Published: (2023)

PersRM-R1: Enhance Personalized Reward Modeling with Reinforcement Learning
by: Li, Mengdi, et al.
Published: (2025)

Pairwise Calibrated Rewards for Pluralistic Alignment
by: Halpern, Daniel, et al.
Published: (2025)

Reward Models in Deep Reinforcement Learning: A Survey
by: Yu, Rui, et al.
Published: (2025)

Weight Scope Alignment: A Frustratingly Easy Method for Model Merging
by: Xu, Yichu, et al.
Published: (2024)

Reward-free Alignment for Conflicting Objectives
by: Chen, Peter, et al.
Published: (2026)

On the Robustness of Reward Models for Language Model Alignment
by: Hong, Jiwoo, et al.
Published: (2025)

Memorized Images in Diffusion Models share a Subspace that can be Located and Deleted
by: Chavhan, Ruchika, et al.
Published: (2024)

AI Alignment with Changing and Influenceable Reward Functions
by: Carroll, Micah, et al.
Published: (2024)

Reward-Augmented Data Enhances Direct Preference Alignment of LLMs
by: Zhang, Shenao, et al.
Published: (2024)

Towards Improving Reward Design in RL: A Reward Alignment Metric for RL Practitioners
by: Muslimani, Calarina, et al.
Published: (2025)

Evidentially Calibrated Source-Free Time-Series Domain Adaptation with Temporal Imputation
by: Ragab, Mohamed, et al.
Published: (2024)

Fine Structure-Aware Sampling: A New Sampling Training Scheme for Pixel-Aligned Implicit Models in Single-View Human Reconstruction
by: Chan, Kennard Yanting, et al.
Published: (2024)

Diamond Maps: Efficient Reward Alignment via Stochastic Flow Maps
by: Holderrieth, Peter, et al.
Published: (2026)

Personalized Federated Learning for Spatio-Temporal Forecasting: A Dual Semantic Alignment-Based Contrastive Approach
by: Liu, Qingxiang, et al.
Published: (2024)

Tiered Reward: Designing Rewards for Specification and Fast Learning of Desired Behavior
by: Zhou, Zhiyuan, et al.
Published: (2022)

Rethinking the Role of Proxy Rewards in Language Model Alignment
by: Kim, Sungdong, et al.
Published: (2024)

ORSO: Accelerating Reward Design via Online Reward Selection and Policy Optimization
by: Zhang, Chen Bo Calvin, et al.
Published: (2024)

Which Rewards Matter? Reward Selection for Reinforcement Learning under Limited Feedback
by: Chaudhari, Shreyas, et al.
Published: (2025)

Reward-Directed Score-Based Diffusion Models via q-Learning
by: Gao, Xuefeng, et al.
Published: (2024)