Imagen de Portada

Direct Preference Optimization With Unobserved Preference Heterogeneity: The Necessity of Ternary Preferences

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Chidambaram, Keertana, Seetharaman, Karthik Vinay, Syrgkanis, Vasilis
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Direct Preference Optimization with Unobserved Preference Heterogeneity: The Necessity of Ternary Preferences
por: Chidambaram, Keertana, et al.
Publicado: (2025)

Sequential Decision Making with Expert Demonstrations under Unobserved Heterogeneity
por: Balazadeh, Vahid, et al.
Publicado: (2024)

Personalized Adaptation via In-Context Preference Learning
por: Lau, Allison, et al.
Publicado: (2024)

Preference Learning with Response Time: Robust Losses and Guarantees
por: Sawarni, Ayush, et al.
Publicado: (2025)

Estimation of Treatment Effects in Extreme and Unobserved Data
por: Tan, Jiyuan, et al.
Publicado: (2025)

Direct Alignment with Heterogeneous Preferences
por: Shirali, Ali, et al.
Publicado: (2025)

Distributed Direct Preference Optimization
por: Jiang, Zhanhong
Publicado: (2026)

Gradient Imbalance in Direct Preference Optimization
por: Ma, Qinwei, et al.
Publicado: (2025)

Active Learning for Direct Preference Optimization
por: Kveton, Branislav, et al.
Publicado: (2025)

Lightweight Robust Direct Preference Optimization
por: Kim, Cheol Woo, et al.
Publicado: (2025)

A Survey of Direct Preference Optimization
por: Liu, Shunyu, et al.
Publicado: (2025)

DIPPER: Direct Preference Optimization to Accelerate Primitive-Enabled Hierarchical Reinforcement Learning
por: Singh, Utsav, et al.
Publicado: (2024)

Right Now, Wrong Then: Non-Stationary Direct Preference Optimization under Preference Drift
por: Son, Seongho, et al.
Publicado: (2024)

Filtered Direct Preference Optimization
por: Morimura, Tetsuro, et al.
Publicado: (2024)

Direct Preference Optimization with an Offset
por: Amini, Afra, et al.
Publicado: (2024)

Length Desensitization in Direct Preference Optimization
por: Liu, Wei, et al.
Publicado: (2024)

ADPO: Anchored Direct Preference Optimization
por: Zixian, Wang
Publicado: (2025)

Continuous-Utility Direct Preference Optimization
por: Mohsin, Muhammad Ahmed, et al.
Publicado: (2026)

Data-Centric Human Preference with Rationales for Direct Preference Alignment
por: Just, Hoang Anh, et al.
Publicado: (2024)

Direct Preference Optimization for Primitive-Enabled Hierarchical RL: A Bilevel Approach
por: Singh, Utsav, et al.
Publicado: (2024)

Uncertainty-Penalized Direct Preference Optimization
por: Houliston, Sam, et al.
Publicado: (2024)

Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization
por: Zhou, Zhanhui, et al.
Publicado: (2023)

Listwise Direct Preference Optimization with Multi-Dimensional Preference Mixing
por: Sun, Yuhui, et al.
Publicado: (2025)

On the Role of Preference Variance in Preference Optimization
por: Guo, Jiacheng, et al.
Publicado: (2025)

Entropy Controllable Direct Preference Optimization
por: Omura, Motoki, et al.
Publicado: (2024)

Orthogonal Finetuning for Direct Preference Optimization
por: Yang, Chenxu, et al.
Publicado: (2024)

Adaptive Estimation and Inference in Conditional Moment Models via the Discrepancy Principle
por: Tan, Jiyuan, et al.
Publicado: (2026)

A Meta-learner for Heterogeneous Effects in Difference-in-Differences
por: Lan, Hui, et al.
Publicado: (2025)

Understanding Reference Policies in Direct Preference Optimization
por: Liu, Yixin, et al.
Publicado: (2024)

Aligning CodeLLMs with Direct Preference Optimization
por: Miao, Yibo, et al.
Publicado: (2024)

Accelerating Direct Preference Optimization with Prefix Sharing
por: Wang, Franklin, et al.
Publicado: (2024)

Direct Preference Optimization for Adaptive Concept-based Explanations
por: Teneggi, Jacopo, et al.
Publicado: (2025)

Understanding the Impact of Sampling Quality in Direct Preference Optimization
por: Kim, Kyung Rok, et al.
Publicado: (2025)

Post Reinforcement Learning Inference
por: Syrgkanis, Vasilis, et al.
Publicado: (2023)

Preference Optimization with Multi-Sample Comparisons
por: Wang, Chaoqi, et al.
Publicado: (2024)

Preference as Reward, Maximum Preference Optimization with Importance Sampling
por: Jiang, Zaifan, et al.
Publicado: (2023)

DOPL: Direct Online Preference Learning for Restless Bandits with Preference Feedback
por: Xiong, Guojun, et al.
Publicado: (2024)

The Partial Testimony of Logs: Evaluation of Language Model Generation under Confounded Model Choice
por: Jin, Jikai, et al.
Publicado: (2026)

Direct Multi-Turn Preference Optimization for Language Agents
por: Shi, Wentao, et al.
Publicado: (2024)

$β$-DPO: Direct Preference Optimization with Dynamic $β$
por: Wu, Junkang, et al.
Publicado: (2024)