Cover Image

DPO-Shift: Shifting the Distribution of Direct Preference Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Xiliang, Jiang, Feng, Zhang, Qianen, Zhao, Lei, Li, Xiao
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

InCo-DPO: Balancing Distribution Shift and Data Quality for Enhanced Preference Optimization
by: Wang, Yunan, et al.
Published: (2025)

Cal-DPO: Calibrated Direct Preference Optimization for Language Model Alignment
by: Xiao, Teng, et al.
Published: (2024)

Redefining Machine Simultaneous Interpretation: From Incremental Translation to Human-Like Strategies
by: Zhang, Qianen, et al.
Published: (2025)

Online DPO: Online Direct Preference Optimization with Fast-Slow Chasing
by: Qi, Biqing, et al.
Published: (2024)

GroupDPO: Memory efficient Group-wise Direct Preference Optimization
by: Leng, Jixuan, et al.
Published: (2026)

AlphaDPO: Adaptive Reward Margin for Direct Preference Optimization
by: Wu, Junkang, et al.
Published: (2024)

Stable Preference Optimization: A Bilevel Approach to Catastrophic Preference Shift
by: Jian, Chengtao, et al.
Published: (2025)

Redefining Machine Simultaneous Interpretation: From Incremental Translation to Human-Like Strategies
by: Zhang, Qianen, et al.
Published: (2026)

BiasDPO: Mitigating Bias in Language Models through Direct Preference Optimization
by: Allam, Ahmed
Published: (2024)

2D-DPO: Scaling Direct Preference Optimization with 2-Dimensional Supervision
by: Li, Shilong, et al.
Published: (2024)

MCM-DPO: Multifaceted Cross-Modal Direct Preference Optimization for Alt-text Generation
by: Fu, Jinlan, et al.
Published: (2025)

DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimization
by: Zhou, Zhenglin, et al.
Published: (2025)

Pre-DPO: Improving Data Utilization in Direct Preference Optimization Using a Guiding Reference Model
by: Pan, Junshu, et al.
Published: (2025)

AdaDPO: Self-Adaptive Direct Preference Optimization with Balanced Gradient Updates
by: Chen, Shaolong, et al.
Published: (2026)

Full-Step-DPO: Self-Supervised Preference Optimization with Step-wise Rewards for Mathematical Reasoning
by: Xu, Huimin, et al.
Published: (2025)

Robust Prompt Optimization for Large Language Models Against Distribution Shifts
by: Li, Moxin, et al.
Published: (2023)

Proximalized Preference Optimization for Diverse Feedback Types: A Decomposed Perspective on DPO
by: Guo, Kaiyang, et al.
Published: (2025)

Leveraging Robust Optimization for LLM Alignment under Distribution Shifts
by: Zhu, Mingye, et al.
Published: (2025)

Mix- and MoE-DPO: A Variational Inference Approach to Direct Preference Optimization
by: Bohne, Jason, et al.
Published: (2025)

RS-DPO: A Hybrid Rejection Sampling and Direct Preference Optimization Method for Alignment of Large Language Models
by: Khaki, Saeed, et al.
Published: (2024)

DiaTool-DPO: Multi-Turn Direct Preference Optimization for Tool-Augmented Large Language Models
by: Jung, Sunghee, et al.
Published: (2025)

VERI-DPO: Evidence-Aware Alignment for Clinical Summarization via Claim Verification and Direct Preference Optimization
by: Liu, Weixin, et al.
Published: (2026)

TIS-DPO: Token-level Importance Sampling for Direct Preference Optimization With Estimated Weights
by: Liu, Aiwei, et al.
Published: (2024)

Uni-DPO: A Unified Paradigm for Dynamic Preference Optimization of LLMs
by: Peng, Shangpin, et al.
Published: (2025)

SignDPO: Multi-level Direct Preference Optimisation for Skeleton-based Gloss-free Sign Language Translation
by: Pu, Muxin, et al.
Published: (2026)

No Preference Left Behind: Group Distributional Preference Optimization
by: Yao, Binwei, et al.
Published: (2024)

Multi-Preference Optimization: Generalizing DPO via Set-Level Contrasts
by: Gupta, Taneesh, et al.
Published: (2024)

DPO Meets PPO: Reinforced Token Optimization for RLHF
by: Zhong, Han, et al.
Published: (2024)

Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs
by: Lai, Xin, et al.
Published: (2024)

mDPO: Conditional Preference Optimization for Multimodal Large Language Models
by: Wang, Fei, et al.
Published: (2024)

Robust Multi-Objective Preference Alignment with Online DPO
by: Gupta, Raghav, et al.
Published: (2025)

Reducing Hallucination in Vision-Language Models via Stage-wise Preference Optimization under Distribution Shift
by: Xu, Qinwu
Published: (2026)

Ambiguity Awareness Optimization: Towards Semantic Disambiguation for Direct Preference Optimization
by: Li, Jian, et al.
Published: (2025)

DPO Kernels: A Semantically-Aware, Kernel-Enhanced, and Divergence-Rich Paradigm for Direct Preference Optimization
by: Das, Amitava, et al.
Published: (2025)

Sem-DPO: Mitigating Semantic Inconsistency in Preference Optimization for Prompt Engineering
by: Mohamed, Anas, et al.
Published: (2025)

When In-Distribution Gains Fail: Evaluating Weak-to-Strong Reward Models under Preference Shift
by: Le, Khoi, et al.
Published: (2026)

daDPO: Distribution-Aware DPO for Distilling Conversational Abilities
by: Zhang, Zhengze, et al.
Published: (2025)

BPO: Revisiting Preference Modeling in Direct Preference Optimization
by: Sun, Lin, et al.
Published: (2025)

MetaRM: Shifted Distributions Alignment via Meta-Learning
by: Dou, Shihan, et al.
Published: (2024)

A Comprehensive Survey of Direct Preference Optimization: Datasets, Theories, Variants, and Applications
by: Xiao, Wenyi, et al.
Published: (2024)