Imagen de Portada

Margin Adaptive DPO: Leveraging Reward Model for Granular Control in Preference Optimization

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autor principal:	Rho, Hyung Gyu
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

AlphaDPO: Adaptive Reward Margin for Direct Preference Optimization
por: Wu, Junkang, et al.
Publicado: (2024)

$ξ$-DPO: Direct Preference Optimization via Ratio Reward Margin
por: Fan, Zhengyuan, et al.
Publicado: (2026)

Best of mini-N in-loop Sampling: A Contextual Quality Reward Model for Reliable and Efficient Best-of-N Sampling
por: Rho, Hyung Gyu, et al.
Publicado: (2025)

C2-DPO: Constrained Controlled Direct Preference Optimization
por: Asadi, Kavosh, et al.
Publicado: (2025)

$β$-DPO: Direct Preference Optimization with Dynamic $β$
por: Wu, Junkang, et al.
Publicado: (2024)

Difficulty-Based Preference Data Selection by DPO Implicit Reward Gap
por: Qi, Xuan, et al.
Publicado: (2025)

ActiveDPO: Active Direct Preference Optimization for Sample-Efficient Alignment
por: Lin, Xiaoqiang, et al.
Publicado: (2025)

SafeDPO: A Simple Approach to Direct Preference Optimization with Enhanced Safety
por: Kim, Geon-Hyeong, et al.
Publicado: (2025)

AMoPO: Adaptive Multi-objective Preference Optimization without Reward Models and Reference Models
por: Liu, Qi, et al.
Publicado: (2025)

Larger or Smaller Reward Margins to Select Preferences for Alignment?
por: Huang, Kexin, et al.
Publicado: (2025)

Preference as Reward, Maximum Preference Optimization with Importance Sampling
por: Jiang, Zaifan, et al.
Publicado: (2023)

Adaptive Margin RLHF via Preference over Preferences
por: Chittepu, Yaswanth, et al.
Publicado: (2025)

MixDPO: Modeling Preference Strength for Pluralistic Alignment
por: Imai, Saki, et al.
Publicado: (2026)

Sem-DPO: Mitigating Semantic Inconsistency in Preference Optimization for Prompt Engineering
por: Mohamed, Anas, et al.
Publicado: (2025)

Online DPO: Online Direct Preference Optimization with Fast-Slow Chasing
por: Qi, Biqing, et al.
Publicado: (2024)

Multi-Preference Optimization: Generalizing DPO via Set-Level Contrasts
por: Gupta, Taneesh, et al.
Publicado: (2024)

MDPO: Multi-Granularity Direct Preference Optimization for Mathematical Reasoning
por: Lin, Yunze
Publicado: (2025)

EnerBridge-DPO: Energy-Guided Protein Inverse Folding with Markov Bridges and Direct Preference Optimization
por: Rong, Dingyi, et al.
Publicado: (2025)

Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs
por: Lai, Xin, et al.
Publicado: (2024)

APLOT: Robust Reward Modeling via Adaptive Preference Learning with Optimal Transport
por: Li, Zhuo, et al.
Publicado: (2025)

GFRIEND: Generative Few-shot Reward Inference through EfficieNt DPO
por: Zhao, Yiyang, et al.
Publicado: (2025)

Mix- and MoE-DPO: A Variational Inference Approach to Direct Preference Optimization
por: Bohne, Jason, et al.
Publicado: (2025)

mDPO: Conditional Preference Optimization for Multimodal Large Language Models
por: Wang, Fei, et al.
Publicado: (2024)

In-Context Reward Adaptation for Robust Preference Modeling
por: Sun, Zhenyu, et al.
Publicado: (2026)

Curriculum-DPO++: Direct Preference Optimization via Data and Model Curricula for Text-to-Image Generation
por: Croitoru, Florinel-Alin, et al.
Publicado: (2026)

Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive
por: Pal, Arka, et al.
Publicado: (2024)

Implicit Reward as the Bridge: A Unified View of SFT and DPO Connections
por: Wang, Bo, et al.
Publicado: (2025)

PhysMoDPO: Physically-Plausible Humanoid Motion with Preference Optimization
por: Zhang, Yangsong, et al.
Publicado: (2026)

PaTaRM: Bridging Pairwise and Pointwise Signals via Preference-Aware Task-Adaptive Reward Modeling
por: Jian, Ai, et al.
Publicado: (2025)

Curry-DPO: Enhancing Alignment using Curriculum Learning & Ranked Preferences
por: Pattnaik, Pulkit, et al.
Publicado: (2024)

Direct Preference Optimization: Your Language Model is Secretly a Reward Model
por: Rafailov, Rafael, et al.
Publicado: (2023)

Uni-DPO: A Unified Paradigm for Dynamic Preference Optimization of LLMs
por: Peng, Shangpin, et al.
Publicado: (2025)

IRPM: Intergroup Relative Preference Modeling for Pointwise Generative Reward Models
por: Song, Haonan, et al.
Publicado: (2026)

Optimal Transport for LLM Reward Modeling from Noisy Preference
por: Pan, Licheng, et al.
Publicado: (2026)

T-REG: Preference Optimization with Token-Level Reward Regularization
por: Zhou, Wenxuan, et al.
Publicado: (2024)

VPO: Leveraging the Number of Votes in Preference Optimization
por: Cho, Jae Hyeon, et al.
Publicado: (2024)

Reward Learning From Preference With Ties
por: Liu, Jinsong, et al.
Publicado: (2024)

PROF: An LLM-based Reward Code Preference Optimization Framework for Offline Imitation Learning
por: Sun, Shengjie, et al.
Publicado: (2025)

$i$REPO: $i$mplicit Reward Pairwise Difference based Empirical Preference Optimization
por: Le, Long Tan, et al.
Publicado: (2024)

What Matters in Data for DPO?
por: Pan, Yu, et al.
Publicado: (2025)