Image de couverture de livre

Proximalized Preference Optimization for Diverse Feedback Types: A Decomposed Perspective on DPO

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Guo, Kaiyang, Li, Yinchuan, Chen, Zhitang
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

DPO-Shift: Shifting the Distribution of Direct Preference Optimization
par: Yang, Xiliang, et autres
Publié: (2025)

Unpacking DPO and PPO: Disentangling Best Practices for Learning from Preference Feedback
par: Ivison, Hamish, et autres
Publié: (2024)

Preference Optimization for Reasoning with Pseudo Feedback
par: Jiao, Fangkai, et autres
Publié: (2024)

2D-DPO: Scaling Direct Preference Optimization with 2-Dimensional Supervision
par: Li, Shilong, et autres
Publié: (2024)

GroupDPO: Memory efficient Group-wise Direct Preference Optimization
par: Leng, Jixuan, et autres
Publié: (2026)

Cal-DPO: Calibrated Direct Preference Optimization for Language Model Alignment
par: Xiao, Teng, et autres
Publié: (2024)

Full-Step-DPO: Self-Supervised Preference Optimization with Step-wise Rewards for Mathematical Reasoning
par: Xu, Huimin, et autres
Publié: (2025)

AdaDPO: Self-Adaptive Direct Preference Optimization with Balanced Gradient Updates
par: Chen, Shaolong, et autres
Publié: (2026)

BiasDPO: Mitigating Bias in Language Models through Direct Preference Optimization
par: Allam, Ahmed
Publié: (2024)

Online DPO: Online Direct Preference Optimization with Fast-Slow Chasing
par: Qi, Biqing, et autres
Publié: (2024)

Diverse Preference Optimization
par: Lanchantin, Jack, et autres
Publié: (2025)

MCM-DPO: Multifaceted Cross-Modal Direct Preference Optimization for Alt-text Generation
par: Fu, Jinlan, et autres
Publié: (2025)

AlphaDPO: Adaptive Reward Margin for Direct Preference Optimization
par: Wu, Junkang, et autres
Publié: (2024)

InCo-DPO: Balancing Distribution Shift and Data Quality for Enhanced Preference Optimization
par: Wang, Yunan, et autres
Publié: (2025)

mDPO: Conditional Preference Optimization for Multimodal Large Language Models
par: Wang, Fei, et autres
Publié: (2024)

Robust Multi-Objective Preference Alignment with Online DPO
par: Gupta, Raghav, et autres
Publié: (2025)

Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs
par: Lai, Xin, et autres
Publié: (2024)

Uni-DPO: A Unified Paradigm for Dynamic Preference Optimization of LLMs
par: Peng, Shangpin, et autres
Publié: (2025)

Sem-DPO: Mitigating Semantic Inconsistency in Preference Optimization for Prompt Engineering
par: Mohamed, Anas, et autres
Publié: (2025)

Multi-Preference Optimization: Generalizing DPO via Set-Level Contrasts
par: Gupta, Taneesh, et autres
Publié: (2024)

DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimization
par: Zhou, Zhenglin, et autres
Publié: (2025)

Pre-DPO: Improving Data Utilization in Direct Preference Optimization Using a Guiding Reference Model
par: Pan, Junshu, et autres
Publié: (2025)

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment
par: Cho, Jay Hyeon, et autres
Publié: (2025)

Mix- and MoE-DPO: A Variational Inference Approach to Direct Preference Optimization
par: Bohne, Jason, et autres
Publié: (2025)

Towards Analyzing and Understanding the Limitations of DPO: A Theoretical Perspective
par: Feng, Duanyu, et autres
Publié: (2024)

Understanding the Performance Gap in Preference Learning: A Dichotomy of RLHF and DPO
par: Shi, Ruizhe, et autres
Publié: (2025)

Provably Robust DPO: Aligning Language Models with Noisy Feedback
par: Chowdhury, Sayak Ray, et autres
Publié: (2024)

PersoDPO: Scalable Preference Optimization for Instruction-Adherent, Persona-Grounded Dialogue via Multi-LLM Evaluation
par: Afzoon, Saleh, et autres
Publié: (2026)

DiaTool-DPO: Multi-Turn Direct Preference Optimization for Tool-Augmented Large Language Models
par: Jung, Sunghee, et autres
Publié: (2025)

VERI-DPO: Evidence-Aware Alignment for Clinical Summarization via Claim Verification and Direct Preference Optimization
par: Liu, Weixin, et autres
Publié: (2026)

MixDPO: Modeling Preference Strength for Pluralistic Alignment
par: Imai, Saki, et autres
Publié: (2026)

TIS-DPO: Token-level Importance Sampling for Direct Preference Optimization With Estimated Weights
par: Liu, Aiwei, et autres
Publié: (2024)

RS-DPO: A Hybrid Rejection Sampling and Direct Preference Optimization Method for Alignment of Large Language Models
par: Khaki, Saeed, et autres
Publié: (2024)

Test-Time Preference Optimization: On-the-Fly Alignment via Iterative Textual Feedback
par: Li, Yafu, et autres
Publié: (2025)

Recovering Diversity Without Losing Alignment: A DPO Recipe for Post-Trained LLMs
par: Samuel, Vinay, et autres
Publié: (2026)

Towards Low-Resource Alignment to Diverse Perspectives with Sparse Feedback
par: Luo, Chu Fei, et autres
Publié: (2025)

Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive
par: Pal, Arka, et autres
Publié: (2024)

Personalizing LLMs with Binary Feedback: A Preference-Corrected Optimization Framework
par: Ma, Xilai, et autres
Publié: (2026)

SignDPO: Multi-level Direct Preference Optimisation for Skeleton-based Gloss-free Sign Language Translation
par: Pu, Muxin, et autres
Publié: (2026)

FinDPO: Financial Sentiment Analysis for Algorithmic Trading through Preference Optimization of LLMs
par: Iacovides, Giorgos, et autres
Publié: (2025)