Cover Image

What Is Preference Optimization Doing, and Why?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Yue, Wang, Qizhou, Zhang, Zizhuo, Niu, Gang, Han, Bo, Sugiyama, Masashi
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Towards Understanding Valuable Preference Data for Large Language Model Alignment
by: Zhang, Zizhuo, et al.
Published: (2025)

Towards Effective Evaluations and Comparisons for LLM Unlearning Methods
by: Wang, Qizhou, et al.
Published: (2024)

Towards Scalable Oversight with Collaborative Multi-Agent Debate in Error Detection
by: Chen, Yongqiang, et al.
Published: (2025)

Decoupling the Class Label and the Target Concept in Machine Unlearning
by: Zhu, Jianing, et al.
Published: (2024)

BrokenBind: Universal Modality Exploration beyond Dataset Boundaries
by: Huang, Zhuo, et al.
Published: (2026)

In-context Demonstration Matters: On Prompt Optimization for Pseudo-Supervision Refinement
by: Zhang, Zhen-Yu, et al.
Published: (2024)

On Symmetric Losses for Robust Policy Optimization with Noisy Preferences
by: Nishimori, Soichiro, et al.
Published: (2025)

Learning with Complementary Labels Revisited: The Selected-Completely-at-Random Setting Is More Practical
by: Wang, Wei, et al.
Published: (2023)

Rethinking Consistent Multi-Label Classification Under Inexact Supervision
by: Wang, Wei, et al.
Published: (2025)

Realistic Evaluation of Deep Partial-Label Learning Algorithms
by: Wang, Wei, et al.
Published: (2025)

Accessible, Realistic, and Fair Evaluation of Positive-Unlabeled Learning Algorithms
by: Wang, Wei, et al.
Published: (2025)

Generating Chain-of-Thoughts with a Pairwise-Comparison Approach to Searching for the Most Promising Intermediate Thought
by: Zhang, Zhen-Yu, et al.
Published: (2024)

Balancing Similarity and Complementarity for Federated Learning
by: Yan, Kunda, et al.
Published: (2024)

Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers
by: Cai, Xin-Qiang, et al.
Published: (2025)

Accurate Forgetting for Heterogeneous Federated Continual Learning
by: Wuerkaixi, Abudukelimu, et al.
Published: (2025)

A Fast Algorithm for the Real-Valued Combinatorial Pure Exploration of Multi-Armed Bandit
by: Nakamura, Shintaro, et al.
Published: (2023)

What Makes "Good" Distractors for Object Hallucination Evaluation in Large Vision-Language Models?
by: Xie, Ming-Kun, et al.
Published: (2025)

Dual-Decoupling Learning and Metric-Adaptive Thresholding for Semi-Supervised Multi-Label Learning
by: Xiao, Jia-Hao, et al.
Published: (2024)

Enriching Disentanglement: From Logical Definitions to Quantitative Metrics
by: Zhang, Yivan, et al.
Published: (2023)

Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimization
by: Fan, Ziqing, et al.
Published: (2024)

Robust Multi-View Learning via Representation Fusion of Sample-Level Attention and Alignment of Simulated Perturbation
by: Xu, Jie, et al.
Published: (2025)

Sharpness-Aware Black-Box Optimization
by: Ye, Feiyang, et al.
Published: (2024)

Multi-Player Approaches for Dueling Bandits
by: Raveh, Or, et al.
Published: (2024)

A Category-theoretical Meta-analysis of Definitions of Disentanglement
by: Zhang, Yivan, et al.
Published: (2023)

BadLabel: A Robust Perspective on Evaluating and Enhancing Label-noise Learning
by: Zhang, Jingfeng, et al.
Published: (2023)

VEC-SBM: Optimal Community Detection with Vectorial Edges Covariates
by: Braun, Guillaume, et al.
Published: (2024)

Riemannian Langevin Dynamics: Strong Convergence of Geometric Euler-Maruyama Scheme
by: Zhan, Zhiyuan, et al.
Published: (2026)

Non-stationary Online Learning for Curved Losses: Improved Dynamic Regret via Mixability
by: Zhang, Yu-Jie, et al.
Published: (2025)

From Small to Large: A Graph Convolutional Network Approach for Solving Assortment Optimization Problems
by: Li, Guokai, et al.
Published: (2025)

From Coefficients to Directions: Rethinking Model Merging with Directional Alignment
by: Chen, Zhikang, et al.
Published: (2025)

Practical estimation of the optimal classification error with soft labels and calibration
by: Ushio, Ryota, et al.
Published: (2025)

The Survival Bandit Problem
by: Riou, Charles, et al.
Published: (2022)

Thompson Exploration with Best Challenger Rule in Best Arm Identification
by: Lee, Jongyeong, et al.
Published: (2023)

VI-CuRL: Stabilizing Verifier-Independent RL Reasoning via Confidence-Guided Variance Reduction
by: Cai, Xin-Qiang, et al.
Published: (2026)

GRU: Mitigating the Trade-off between Unlearning and Retention for LLMs
by: Wang, Yue, et al.
Published: (2025)

LLM Unlearning with LLM Beliefs
by: Li, Kemou, et al.
Published: (2025)

Reinforcement Learning with Options and State Representation
by: Ghriss, Ayoub, et al.
Published: (2024)

Bifrost: Steering Strategic Trajectories to Bridge Contextual Gaps for Self-Improving Agents
by: Tran, Quan M., et al.
Published: (2026)

LLM Routing with Dueling Feedback
by: Chiang, Chao-Kai, et al.
Published: (2025)

Offline Reinforcement Learning from Datasets with Structured Non-Stationarity
by: Ackermann, Johannes, et al.
Published: (2024)