Cover Image

Robust Preference Optimization via Dynamic Target Margins

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Sun, Jie, Wu, Junkang, Wu, Jiancan, Zhu, Zhibo, Lu, Xingyu, Zhou, Jun, Ma, Lintao, Wang, Xiang
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

AlphaDPO: Adaptive Reward Margin for Direct Preference Optimization
by: Wu, Junkang, et al.
Published: (2024)

Larger or Smaller Reward Margins to Select Preferences for Alignment?
by: Huang, Kexin, et al.
Published: (2025)

Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimization
by: Wu, Junkang, et al.
Published: (2024)

SepSeq: A Training-Free Framework for Long Numerical Sequence Processing in LLMs
by: Sun, Jie, et al.
Published: (2026)

LaMP-Val: Large Language Models Empower Personalized Valuation in Auction
by: Sun, Jie, et al.
Published: (2024)

bi-GRPO: Bidirectional Optimization for Jailbreak Backdoor Injection on LLMs
by: Ji, Wence, et al.
Published: (2025)

AdaViP: Aligning Multi-modal LLMs via Adaptive Vision-enhanced Preference Optimization
by: Lu, Jinda, et al.
Published: (2025)

Direct Multi-Turn Preference Optimization for Language Agents
by: Shi, Wentao, et al.
Published: (2024)

On Negative-aware Preference Optimization for Recommendation
by: Ding, Chenlu, et al.
Published: (2025)

Length-Controlled Margin-Based Preference Optimization without Reference Model
by: Li, Gengxu, et al.
Published: (2025)

$β$-DPO: Direct Preference Optimization with Dynamic $β$
by: Wu, Junkang, et al.
Published: (2024)

BPO: Revisiting Preference Modeling in Direct Preference Optimization
by: Sun, Lin, et al.
Published: (2025)

RoseRAG: Robust Retrieval-augmented Generation with Small-scale LLMs via Margin-aware Preference Optimization
by: Liu, Tianci, et al.
Published: (2025)

R^2-Mem: Reflective Experience for Memory Search
by: Wang, Xinyuan, et al.
Published: (2026)

Enhancing Temporal Sensitivity of Large Language Model for Recommendation with Counterfactual Tuning
by: Liu, Yutian, et al.
Published: (2025)

ROPO: Robust Preference Optimization for Large Language Models
by: Liang, Xize, et al.
Published: (2024)

Offline Preference Optimization via Maximum Marginal Likelihood Estimation
by: Najafi, Saeed, et al.
Published: (2025)

PA-RAG: RAG Alignment via Multi-Perspective Preference Optimization
by: Wu, Jiayi, et al.
Published: (2024)

Self-Steering Optimization: Autonomous Preference Optimization for Large Language Models
by: Xiang, Hao, et al.
Published: (2024)

Mitigating Strategy Preference Bias in Emotional Support Conversation via Uncertainty Estimations
by: Zhou, Yougen, et al.
Published: (2025)

Res-Bench: Benchmarking the Robustness of Multimodal Large Language Models to Dynamic Resolution Input
by: Li, Chenxu, et al.
Published: (2025)

Margin Matching Preference Optimization: Enhanced Model Alignment with Granular Feedback
by: Kim, Kyuyoung, et al.
Published: (2024)

Adaptive Margin RLHF via Preference over Preferences
by: Chittepu, Yaswanth, et al.
Published: (2025)

DynamicPO: Dynamic Preference Optimization for Recommendation
by: Hu, Xingyu, et al.
Published: (2026)

RePO: Understanding Preference Learning Through ReLU-Based Optimization
by: Wu, Junkang, et al.
Published: (2025)

AMaPO: Adaptive Margin-attached Preference Optimization for Language Model Alignment
by: Deng, Ruibo, et al.
Published: (2025)

Reverse Preference Optimization for Complex Instruction Following
by: Huang, Xiang, et al.
Published: (2025)

Confidence Before Answering: A Paradigm Shift for Efficient LLM Uncertainty Estimation
by: Li, Changcheng, et al.
Published: (2026)

LAPO: Internalizing Reasoning Efficiency via Length-Adaptive Policy Optimization
by: Wu, Xingyu, et al.
Published: (2025)

Token-level Direct Preference Optimization
by: Zeng, Yongcheng, et al.
Published: (2024)

DuPO: Enabling Reliable LLM Self-Verification via Dual Preference Optimization
by: She, Shuaijie, et al.
Published: (2025)

Decoupling Strategy and Execution in Task-Focused Dialogue via Goal-Oriented Preference Optimization
by: Xu, Jingyi, et al.
Published: (2026)

On the Role of Preference Variance in Preference Optimization
by: Guo, Jiacheng, et al.
Published: (2025)

SeaPO: Strategic Error Amplification for Robust Preference Optimization of Large Language Models
by: Rao, Jun, et al.
Published: (2025)

InfiFPO: Implicit Model Fusion via Preference Optimization in Large Language Models
by: Gu, Yanggan, et al.
Published: (2025)

MAPO: Advancing Multilingual Reasoning through Multilingual Alignment-as-Preference Optimization
by: She, Shuaijie, et al.
Published: (2024)

Omni-RRM: Advancing Omni Reward Modeling via Automatic Rubric-Grounded Preference Synthesis
by: Kong, Zicheng, et al.
Published: (2026)

Direct Judgement Preference Optimization
by: Wang, Peifeng, et al.
Published: (2024)

MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning
by: Li, Chengpeng, et al.
Published: (2023)

Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization
by: Wang, Weiyun, et al.
Published: (2024)