Cover Image

Robust LLM Alignment via Distributionally Robust Direct Preference Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xu, Zaiyan, Vemuri, Sushil, Panaganti, Kishan, Kalathil, Dileep, Jain, Rahul, Ramachandran, Deepak
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Group Distributionally Robust Optimization-Driven Reinforcement Learning for LLM Reasoning
by: Panaganti, Kishan, et al.
Published: (2026)

Efficient Policy Optimization in Robust Constrained MDPs with Iteration Complexity Guarantees
by: Ganguly, Sourav, et al.
Published: (2025)

Adaptive Outer-Loop Control of Quadrotors via Reinforcement Learning
by: Saj, Vishnu, et al.
Published: (2026)

Federated Ensemble-Directed Offline Reinforcement Learning
by: Rengarajan, Desik, et al.
Published: (2023)

Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimization
by: Wu, Junkang, et al.
Published: (2024)

Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoning
by: Parashar, Shubham, et al.
Published: (2025)

Adversarial Preference Learning for Robust LLM Alignment
by: Wang, Yuanfu, et al.
Published: (2025)

Off-Policy Evaluation Using Information Borrowing and Context-Based Switching
by: Dasgupta, Sutanoy, et al.
Published: (2021)

Multi-Objective Preference Optimization: Improving Human Alignment of Generative Models
by: Agnihotri, Akhil, et al.
Published: (2025)

Balance Equation-based Distributionally Robust Offline Imitation Learning
by: Agrawal, Rishabh, et al.
Published: (2025)

KL-regularization Itself is Differentially Private in Bandits and RLHF
by: Zhang, Yizhou, et al.
Published: (2025)

Can LLMs Guide Their Own Exploration? Gradient-Guided Reinforcement Learning for LLM Reasoning
by: Liang, Zhenwen, et al.
Published: (2025)

Distributionally Robust Cooperative Multi-Agent Reinforcement Learning via Robust Value Factorization
by: Qu, Chengrui, et al.
Published: (2026)

Self-Improving Robust Preference Optimization
by: Choi, Eugene, et al.
Published: (2024)

Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization
by: Zhou, Zhanhui, et al.
Published: (2023)

Direct Alignment with Heterogeneous Preferences
by: Shirali, Ali, et al.
Published: (2025)

Inducing Robustness in a 2 Dimensional Direct Preference Optimization Paradigm
by: Shashidhar, Sarvesh, et al.
Published: (2025)

Spread Preference Annotation: Direct Preference Judgment for Efficient LLM Alignment
by: Kim, Dongyoung, et al.
Published: (2024)

Model-Free Robust $ϕ$-Divergence Reinforcement Learning Using Both Offline and Online Data
by: Panaganti, Kishan, et al.
Published: (2024)

ActiveDPO: Active Direct Preference Optimization for Sample-Efficient Alignment
by: Lin, Xiaoqiang, et al.
Published: (2025)

Similarity as Reward Alignment: Robust and Versatile Preference-based Reinforcement Learning
by: Rajaram, Sara, et al.
Published: (2025)

On Symmetric Losses for Robust Policy Optimization with Noisy Preferences
by: Nishimori, Soichiro, et al.
Published: (2025)

When Dynamics Shift, Robust Task Inference Wins: Offline Imitation Learning with Behavior Foundation Models Revisited
by: Agrawal, Rishabh, et al.
Published: (2026)

Conformal Feedback Alignment: Quantifying Answer-Level Reliability for Robust LLM Alignment
by: Chen, Tiejin, et al.
Published: (2026)

In-Context Reward Adaptation for Robust Preference Modeling
by: Sun, Zhenyu, et al.
Published: (2026)

Distributionally Robust Token Optimization in RLHF
by: Jin, Yeping, et al.
Published: (2026)

Adversarial Preference Optimization: Enhancing Your Alignment via RM-LLM Game
by: Cheng, Pengyu, et al.
Published: (2023)

Stochastic Re-weighted Gradient Descent via Distributionally Robust Optimization
by: Kumar, Ramnath, et al.
Published: (2023)

FairPO: Robust Preference Optimization for Fair Multi-Label Learning
by: Mondal, Soumen Kumar, et al.
Published: (2025)

Structured Reinforcement Learning for Media Streaming at the Wireless Edge
by: Bura, Archana, et al.
Published: (2024)

Optimistic World Models: Efficient Exploration in Model-Based Deep Reinforcement Learning
by: Mete, Akshay, et al.
Published: (2026)

Distributionally Robust Constrained Reinforcement Learning under Strong Duality
by: Zhang, Zhengfei, et al.
Published: (2024)

Distribution Preference Optimization: A Fine-grained Perspective for LLM Unlearning
by: Qin, Kai, et al.
Published: (2025)

KL Penalty Control via Perturbation for Direct Preference Optimization
by: Lee, Sangkyu, et al.
Published: (2025)

ECLIPTICA -- A Framework for Switchable LLM Alignment via CITA - Contrastive Instruction-Tuned Alignment
by: Wanaskar, Kapil, et al.
Published: (2026)

Guided Self-Evolving LLMs with Minimal Human Supervision
by: Yu, Wenhao, et al.
Published: (2025)

Meta-Aligner: Bidirectional Preference-Policy Optimization for Multi-Objective LLMs Alignment
by: Xu, Wenzhe, et al.
Published: (2026)

Doubly Robust Alignment for Large Language Models
by: Xu, Erhan, et al.
Published: (2025)

Reward-Augmented Data Enhances Direct Preference Alignment of LLMs
by: Zhang, Shenao, et al.
Published: (2024)

ADPO: Anchored Direct Preference Optimization
by: Zixian, Wang
Published: (2025)