Image de couverture de livre

Intelligently Weighting Multiple Reference Models for Direct Preference Optimization of LLMs

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wu, Skyler, Echarghaoui, Aymen
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

BALAR : A Bayesian Agentic Loop for Active Reasoning
par: Echarghaoui, Aymen, et autres
Publié: (2026)

Aligning CodeLLMs with Direct Preference Optimization
par: Miao, Yibo, et autres
Publié: (2024)

Missing Data Multiple Imputation for Tabular Q-Learning in Online RL
par: Chasalow, Kyla, et autres
Publié: (2025)

$β$-DPO: Direct Preference Optimization with Dynamic $β$
par: Wu, Junkang, et autres
Publié: (2024)

ADPO: Anchored Direct Preference Optimization
par: Zixian, Wang
Publié: (2025)

Continuous-Utility Direct Preference Optimization
par: Mohsin, Muhammad Ahmed, et autres
Publié: (2026)

Mitigating Mismatch within Reference-based Preference Optimization
par: Yuan, Suqin, et autres
Publié: (2026)

$ξ$-DPO: Direct Preference Optimization via Ratio Reward Margin
par: Fan, Zhengyuan, et autres
Publié: (2026)

CuDIP: Enhancing Theorem Proving in LLMs via Curriculum Learning-based Direct Preference Optimization
par: Shi, Shuming, et autres
Publié: (2025)

AMoPO: Adaptive Multi-objective Preference Optimization without Reward Models and Reference Models
par: Liu, Qi, et autres
Publié: (2025)

Reward-Augmented Data Enhances Direct Preference Alignment of LLMs
par: Zhang, Shenao, et autres
Publié: (2024)

Orthogonal Finetuning for Direct Preference Optimization
par: Yang, Chenxu, et autres
Publié: (2024)

Filtered Direct Preference Optimization
par: Morimura, Tetsuro, et autres
Publié: (2024)

Direct Preference Optimization with an Offset
par: Amini, Afra, et autres
Publié: (2024)

Clear Preferences Leave Traces: Reference Model-Guided Sampling for Preference Learning
par: Diwan, Nirav, et autres
Publié: (2025)

Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization
par: Zhou, Zhanhui, et autres
Publié: (2023)

Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimization
par: Wu, Junkang, et autres
Publié: (2024)

Bootstrapping LLMs via Preference-Based Policy Optimization
par: Jia, Chen
Publié: (2025)

Aligning Large Vision-Language Models by Deep Reinforcement Learning and Direct Preference Optimization
par: Nguyen, Thanh Thi, et autres
Publié: (2025)

WPO: Enhancing RLHF with Weighted Preference Optimization
par: Zhou, Wenxuan, et autres
Publié: (2024)

Adaptive Batch-Wise Sample Scheduling for Direct Preference Optimization
par: Huang, Zixuan, et autres
Publié: (2025)

KL Penalty Control via Perturbation for Direct Preference Optimization
par: Lee, Sangkyu, et autres
Publié: (2025)

MDPO: Multi-Granularity Direct Preference Optimization for Mathematical Reasoning
par: Lin, Yunze
Publié: (2025)

C2-DPO: Constrained Controlled Direct Preference Optimization
par: Asadi, Kavosh, et autres
Publié: (2025)

AlphaDPO: Adaptive Reward Margin for Direct Preference Optimization
par: Wu, Junkang, et autres
Publié: (2024)

Entropy Controllable Direct Preference Optimization
par: Omura, Motoki, et autres
Publié: (2024)

HiPO: Hierarchical Preference Optimization for Adaptive Reasoning in LLMs
par: Kachroo, Darsh, et autres
Publié: (2026)

ActiveDPO: Active Direct Preference Optimization for Sample-Efficient Alignment
par: Lin, Xiaoqiang, et autres
Publié: (2025)

Robust LLM Alignment via Distributionally Robust Direct Preference Optimization
par: Xu, Zaiyan, et autres
Publié: (2025)

Risk-aware Direct Preference Optimization under Nested Risk Measure
par: Zhang, Lijun, et autres
Publié: (2025)

Forward versus Backward: Comparing Reasoning Objectives in Direct Preference Optimization
par: Nikzad, Murtaza, et autres
Publié: (2026)

Direct Preference Optimization: Your Language Model is Secretly a Reward Model
par: Rafailov, Rafael, et autres
Publié: (2023)

SafeDPO: A Simple Approach to Direct Preference Optimization with Enhanced Safety
par: Kim, Geon-Hyeong, et autres
Publié: (2025)

Improving Inverse Folding for Peptide Design with Diversity-regularized Direct Preference Optimization
par: Park, Ryan, et autres
Publié: (2024)

Sharpness-Aware Minimization in Logit Space Efficiently Enhances Direct Preference Optimization
par: Luo, Haocheng, et autres
Publié: (2026)

SGDPO: Self-Guided Direct Preference Optimization for Language Model Alignment
par: Zhu, Wenqiao, et autres
Publié: (2025)

Uncertainty-Penalized Direct Preference Optimization
par: Houliston, Sam, et autres
Publié: (2024)

Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards
par: Wang, Haoxiang, et autres
Publié: (2024)

Direct Alignment with Heterogeneous Preferences
par: Shirali, Ali, et autres
Publié: (2025)

Meta-Aligner: Bidirectional Preference-Policy Optimization for Multi-Objective LLMs Alignment
par: Xu, Wenzhe, et autres
Publié: (2026)