Cover Image

Direct Preference Optimization with Unobserved Preference Heterogeneity: The Necessity of Ternary Preferences

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chidambaram, Keertana, Seetharaman, Karthik Vinary, Syrgkanis, Vasilis
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Direct Preference Optimization With Unobserved Preference Heterogeneity: The Necessity of Ternary Preferences
by: Chidambaram, Keertana, et al.
Published: (2024)

Sequential Decision Making with Expert Demonstrations under Unobserved Heterogeneity
by: Balazadeh, Vahid, et al.
Published: (2024)

Preference Learning with Response Time: Robust Losses and Guarantees
by: Sawarni, Ayush, et al.
Published: (2025)

Personalized Adaptation via In-Context Preference Learning
by: Lau, Allison, et al.
Published: (2024)

Direct Alignment with Heterogeneous Preferences
by: Shirali, Ali, et al.
Published: (2025)

Autoregressive Direct Preference Optimization
by: Oi, Masanari, et al.
Published: (2026)

Filtered Direct Preference Optimization
by: Morimura, Tetsuro, et al.
Published: (2024)

Direct Preference Optimization with an Offset
by: Amini, Afra, et al.
Published: (2024)

ADPO: Anchored Direct Preference Optimization
by: Zixian, Wang
Published: (2025)

Continuous-Utility Direct Preference Optimization
by: Mohsin, Muhammad Ahmed, et al.
Published: (2026)

Token-level Direct Preference Optimization
by: Zeng, Yongcheng, et al.
Published: (2024)

On Softmax Direct Preference Optimization for Recommendation
by: Chen, Yuxin, et al.
Published: (2024)

Token-Importance Guided Direct Preference Optimization
by: Yang, Ning, et al.
Published: (2025)

Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization
by: Zhou, Zhanhui, et al.
Published: (2023)

On the Role of Preference Variance in Preference Optimization
by: Guo, Jiacheng, et al.
Published: (2025)

Entropy Controllable Direct Preference Optimization
by: Omura, Motoki, et al.
Published: (2024)

Orthogonal Finetuning for Direct Preference Optimization
by: Yang, Chenxu, et al.
Published: (2024)

TODO: Enhancing LLM Alignment with Ternary Preferences
by: Guo, Yuxiang, et al.
Published: (2024)

Aligning CodeLLMs with Direct Preference Optimization
by: Miao, Yibo, et al.
Published: (2024)

Preference as Reward, Maximum Preference Optimization with Importance Sampling
by: Jiang, Zaifan, et al.
Published: (2023)

Unified Preference Optimization: Language Model Alignment Beyond the Preference Frontier
by: Badrinath, Anirudhan, et al.
Published: (2024)

TUR-DPO: Topology- and Uncertainty-Aware Direct Preference Optimization
by: Abdullah, Abdulhady Abas, et al.
Published: (2026)

Improving Safety Alignment via Balanced Direct Preference Optimization
by: Zhao, Shiji, et al.
Published: (2026)

Learning from Noisy Preferences: A Semi-Supervised Learning Approach to Direct Preference Optimization
by: Liu, Xinxin, et al.
Published: (2026)

$β$-DPO: Direct Preference Optimization with Dynamic $β$
by: Wu, Junkang, et al.
Published: (2024)

CAPO: Confidence Aware Preference Optimization Learning for Multilingual Preferences
by: Pokharel, Rhitabrat, et al.
Published: (2025)

Preference Packing: Efficient Preference Optimization for Large Language Models
by: Cho, Jaekyung
Published: (2026)

Uncertainty-Penalized Direct Preference Optimization
by: Houliston, Sam, et al.
Published: (2024)

Thinking Preference Optimization
by: Yang, Wang, et al.
Published: (2025)

Creative Preference Optimization
by: Ismayilzada, Mete, et al.
Published: (2025)

Geometric-Averaged Preference Optimization for Soft Preference Labels
by: Furuta, Hiroki, et al.
Published: (2024)

Spread Preference Annotation: Direct Preference Judgment for Efficient LLM Alignment
by: Kim, Dongyoung, et al.
Published: (2024)

FocalPO: Enhancing Preference Optimizing by Focusing on Correct Preference Rankings
by: Liu, Tong, et al.
Published: (2025)

Direct Preference-Based Evolutionary Multi-Objective Optimization with Dueling Bandit
by: Huang, Tian, et al.
Published: (2023)

Empirical Analysis of Model Selection for Heterogeneous Causal Effect Estimation
by: Mahajan, Divyat, et al.
Published: (2022)

Learning Causal Representations from General Environments: Identifiability and Intrinsic Ambiguity
by: Jin, Jikai, et al.
Published: (2023)

Adaptive Batch-Wise Sample Scheduling for Direct Preference Optimization
by: Huang, Zixuan, et al.
Published: (2025)

KL Penalty Control via Perturbation for Direct Preference Optimization
by: Lee, Sangkyu, et al.
Published: (2025)

MDPO: Multi-Granularity Direct Preference Optimization for Mathematical Reasoning
by: Lin, Yunze
Published: (2025)

SDPO: Segment-Level Direct Preference Optimization for Social Agents
by: Kong, Aobo, et al.
Published: (2025)