Cover Image

How Well Can Preference Optimization Generalize Under Noisy Feedback?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Im, Shawn, Li, Sharon
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Can DPO Learn Diverse Human Values? A Theoretical Scaling Law
by: Im, Shawn, et al.
Published: (2024)

A Unified Understanding and Evaluation of Steering Methods
by: Im, Shawn, et al.
Published: (2025)

Understanding the Learning Dynamics of Alignment with Human Feedback
by: Im, Shawn, et al.
Published: (2024)

Cyclical Entropy Eruption: Entropy Dynamics in Agent Reinforcement Learning
by: Li, Wendi, et al.
Published: (2026)

How Do Transformers Learn to Associate Tokens: Gradient Leading Terms Bring Mechanistic Interpretability
by: Im, Shawn, et al.
Published: (2026)

Understanding Multimodal LLMs Under Distribution Shifts: An Information-Theoretic Approach
by: Oh, Changdae, et al.
Published: (2025)

On Symmetric Losses for Robust Policy Optimization with Noisy Preferences
by: Nishimori, Soichiro, et al.
Published: (2025)

Debiased Recommendation with Noisy Feedback
by: Li, Haoxuan, et al.
Published: (2024)

Can LLMs Learn to Reason Robustly under Noisy Supervision?
by: Yang, Shenzhi, et al.
Published: (2026)

Semi-Supervised Preference Optimization with Limited Feedback
by: Lee, Seonggyun, et al.
Published: (2025)

Towards Interpretability Without Sacrifice: Faithful Dense Layer Decomposition with Mixture of Decoders
by: Oldfield, James, et al.
Published: (2025)

Optimal Clustering from Noisy Binary Feedback
by: Ariu, Kaito, et al.
Published: (2019)

How Well Can Differential Privacy Be Audited in One Run?
by: Keinan, Amit, et al.
Published: (2025)

CANDERE-COACH: Reinforcement Learning from Noisy Feedback
by: Li, Yuxuan, et al.
Published: (2024)

Neural Dueling Bandits: Preference-Based Optimization with Human Feedback
by: Verma, Arun, et al.
Published: (2024)

RIME: Robust Preference-based Reinforcement Learning with Noisy Preferences
by: Cheng, Jie, et al.
Published: (2024)

Combinatorial Reinforcement Learning with Preference Feedback
by: Lee, Joongkyu, et al.
Published: (2025)

How Do the Architecture and Optimizer Affect Representation Learning? On the Training Dynamics of Representations in Deep Neural Networks
by: Sharon, Yuval, et al.
Published: (2024)

Queueing Matching Bandits with Preference Feedback
by: Kim, Jung-hun, et al.
Published: (2024)

How Well Can Transformers Emulate In-context Newton's Method?
by: Giannou, Angeliki, et al.
Published: (2024)

How Useful is Intermittent, Asynchronous Expert Feedback for Bayesian Optimization?
by: Kristiadi, Agustinus, et al.
Published: (2024)

Online Iterative Reinforcement Learning from Human Feedback with General Preference Model
by: Ye, Chenlu, et al.
Published: (2024)

DOPL: Direct Online Preference Learning for Restless Bandits with Preference Feedback
by: Xiong, Guojun, et al.
Published: (2024)

Optimal Transport for LLM Reward Modeling from Noisy Preference
by: Pan, Licheng, et al.
Published: (2026)

Bradley-Terry Policy Optimization for Generative Preference Modeling
by: Feng, Shengyu, et al.
Published: (2025)

FUN-AD: Fully Unsupervised Learning for Anomaly Detection with Noisy Training Data
by: Im, Jiin, et al.
Published: (2024)

Extragradient Preference Optimization (EGPO): Beyond Last-Iterate Convergence for Nash Learning from Human Feedback
by: Zhou, Runlong, et al.
Published: (2025)

Adaptive Preference Scaling for Reinforcement Learning with Human Feedback
by: Hong, Ilgee, et al.
Published: (2024)

From Noisy Traces to Stable Gradients: Bias-Variance Optimized Preference Optimization for Aligning Large Reasoning Models
by: Zhu, Mingkang, et al.
Published: (2025)

Reducing Oracle Feedback with Vision-Language Embeddings for Preference-Based RL
by: Ghosh, Udita, et al.
Published: (2026)

Provably Robust DPO: Aligning Language Models with Noisy Feedback
by: Chowdhury, Sayak Ray, et al.
Published: (2024)

Dynamic Assortment Selection and Pricing with Censored Preference Feedback
by: Kim, Jung-hun, et al.
Published: (2025)

Best Policy Learning from Trajectory Preference Feedback
by: Agnihotri, Akhil, et al.
Published: (2025)

Preference is More Than Comparisons: Rethinking Dueling Bandits with Augmented Human Feedback
by: Wang, Shengbo, et al.
Published: (2025)

Reinforced Interactive Continual Learning via Real-time Noisy Human Feedback
by: Yang, Yutao, et al.
Published: (2025)

Generation from Noisy Examples
by: Raman, Ananth, et al.
Published: (2025)

Adaptive Measurement Allocation for Learning Kernelized SVMs Under Noisy Observations
by: Miroszewski, Artur
Published: (2026)

Can Large Language Models Reason and Optimize Under Constraints?
by: Bernier, Fabien, et al.
Published: (2026)

How Well Does Your Tabular Generator Learn the Structure of Tabular Data?
by: Jiang, Xiangjian, et al.
Published: (2025)

Robust Recommendation from Noisy Implicit Feedback: A GMM-Weighted Bayes-label Transition Matrix Framework
by: Li, Zongyu, et al.
Published: (2026)