Cover Image

Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhou, Zhanhui, Liu, Jie, Shao, Jing, Yue, Xiangyu, Yang, Chao, Ouyang, Wanli, Qiao, Yu
Format:	Preprint
Published:	2023
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Iterative Length-Regularized Direct Preference Optimization: A Case Study on Improving 7B Language Models to GPT-4 Level
by: Liu, Jie, et al.
Published: (2024)

Emulated Disalignment: Safety Alignment for Large Language Models May Backfire!
by: Zhou, Zhanhui, et al.
Published: (2024)

Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment
by: Guo, Yiju, et al.
Published: (2024)

Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards
by: Wang, Haoxiang, et al.
Published: (2024)

Meta-Aligner: Bidirectional Preference-Policy Optimization for Multi-Objective LLMs Alignment
by: Xu, Wenzhe, et al.
Published: (2026)

Direct Preference-Based Evolutionary Multi-Objective Optimization with Dueling Bandit
by: Huang, Tian, et al.
Published: (2023)

Property-driven Protein Inverse Folding With Multi-Objective Preference Alignment
by: Hou, Xiaoyang, et al.
Published: (2026)

Unified Preference Optimization: Language Model Alignment Beyond the Preference Frontier
by: Badrinath, Anirudhan, et al.
Published: (2024)

Inference-Time Language Model Alignment via Integrated Value Guidance
by: Liu, Zhixuan, et al.
Published: (2024)

Preference Orchestrator: Prompt-Aware Multi-Objective Alignment for Large Language Models
by: Liu, Biao, et al.
Published: (2025)

Attacks, Defenses and Evaluations for LLM Conversation Safety: A Survey
by: Dong, Zhichen, et al.
Published: (2024)

Preference-Driven Multi-Objective Combinatorial Optimization with Conditional Computation
by: Fan, Mingfeng, et al.
Published: (2025)

Beyond One-Size-Fits-All: Adapting Counterfactual Explanations to User Objectives
by: Mastromichalakis, Orfeas Menis, et al.
Published: (2024)

Beyond Preferences in AI Alignment
by: Zhi-Xuan, Tan, et al.
Published: (2024)

Orthogonal Finetuning for Direct Preference Optimization
by: Yang, Chenxu, et al.
Published: (2024)

Not All Preferences are What You Need for Post-Training: Selective Alignment Strategy for Preference Optimization
by: Dong, Zhijin
Published: (2025)

Direct Alignment with Heterogeneous Preferences
by: Shirali, Ali, et al.
Published: (2025)

Preference-Guided Diffusion for Multi-Objective Offline Optimization
by: Annadani, Yashas, et al.
Published: (2025)

Improving Safety Alignment via Balanced Direct Preference Optimization
by: Zhao, Shiji, et al.
Published: (2026)

Mitigating Hallucination in Multimodal Large Language Model via Hallucination-targeted Direct Preference Optimization
by: Fu, Yuhan, et al.
Published: (2024)

Forward versus Backward: Comparing Reasoning Objectives in Direct Preference Optimization
by: Nikzad, Murtaza, et al.
Published: (2026)

Enhancing Multilingual Counterfactual Generation through Alignment-as-Preference Optimization
by: Wang, Yilong, et al.
Published: (2026)

Direct Preference Optimization with Unobserved Preference Heterogeneity: The Necessity of Ternary Preferences
by: Chidambaram, Keertana, et al.
Published: (2025)

Preference Ranking Optimization for Human Alignment
by: Song, Feifan, et al.
Published: (2023)

Token-Importance Guided Direct Preference Optimization
by: Yang, Ning, et al.
Published: (2025)

Token-level Direct Preference Optimization
by: Zeng, Yongcheng, et al.
Published: (2024)

Meta-Learning Objectives for Preference Optimization
by: Alfano, Carlo, et al.
Published: (2024)

Self-Play Preference Optimization for Language Model Alignment
by: Wu, Yue, et al.
Published: (2024)

Reward-Augmented Data Enhances Direct Preference Alignment of LLMs
by: Zhang, Shenao, et al.
Published: (2024)

Preference-Agile Multi-Objective Optimization for Real-time Vehicle Dispatching
by: Jin, Jiahuan, et al.
Published: (2026)

HiGR: Efficient Generative Slate Recommendation via Hierarchical Planning and Multi-Objective Preference Alignment
by: Pang, Yunsheng, et al.
Published: (2025)

HPS: Hard Preference Sampling for Human Preference Alignment
by: Zou, Xiandong, et al.
Published: (2025)

Stable Preference Optimization: A Bilevel Approach to Catastrophic Preference Shift
by: Jian, Chengtao, et al.
Published: (2025)

Autoregressive Direct Preference Optimization
by: Oi, Masanari, et al.
Published: (2026)

SGDPO: Self-Guided Direct Preference Optimization for Language Model Alignment
by: Zhu, Wenqiao, et al.
Published: (2025)

Spread Preference Annotation: Direct Preference Judgment for Efficient LLM Alignment
by: Kim, Dongyoung, et al.
Published: (2024)

Aligning CodeLLMs with Direct Preference Optimization
by: Miao, Yibo, et al.
Published: (2024)

ActiveDPO: Active Direct Preference Optimization for Sample-Efficient Alignment
by: Lin, Xiaoqiang, et al.
Published: (2025)

Robust LLM Alignment via Distributionally Robust Direct Preference Optimization
by: Xu, Zaiyan, et al.
Published: (2025)

Interactive Hyperparameter Optimization in Multi-Objective Problems via Preference Learning
by: Giovanelli, Joseph, et al.
Published: (2023)