Cover Image

Provably Robust DPO: Aligning Language Models with Noisy Feedback

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chowdhury, Sayak Ray, Kini, Anush, Natarajan, Nagarajan
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Multi-Preference Optimization: Generalizing DPO via Set-Level Contrasts
by: Gupta, Taneesh, et al.
Published: (2024)

Provably Learning from Language Feedback
by: Xu, Wanqiao, et al.
Published: (2025)

Aligning Language Models with Clinical Expertise: DPO for Heart Failure Nursing Documentation in Critical Care
by: Fan, Junyi, et al.
Published: (2025)

Why DPO is a Misspecified Estimator and How to Fix It
by: Gopalan, Aditya, et al.
Published: (2025)

Bootstrapping Language Models with DPO Implicit Rewards
by: Chen, Changyu, et al.
Published: (2024)

Active Preference Optimization for Sample Efficient RLHF
by: Das, Nirjhar, et al.
Published: (2024)

Towards Aligning Language Models with Textual Feedback
by: Lloret, Saüc Abadal, et al.
Published: (2024)

Robust Multi-Objective Preference Alignment with Online DPO
by: Gupta, Raghav, et al.
Published: (2025)

Cal-DPO: Calibrated Direct Preference Optimization for Language Model Alignment
by: Xiao, Teng, et al.
Published: (2024)

Can Language Models Perform Robust Reasoning in Chain-of-thought Prompting with Noisy Rationales?
by: Zhou, Zhanke, et al.
Published: (2024)

KITE: Kernelized and Information Theoretic Exemplars for In-Context Learning
by: Singh, Vaibhav, et al.
Published: (2025)

A Critical Evaluation of AI Feedback for Aligning Large Language Models
by: Sharma, Archit, et al.
Published: (2024)

DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimization
by: Zhou, Zhenglin, et al.
Published: (2025)

Provable Interactive Learning with Hindsight Instruction Feedback
by: Misra, Dipendra, et al.
Published: (2024)

Peering Through Preferences: Unraveling Feedback Acquisition for Aligning Large Language Models
by: Bansal, Hritik, et al.
Published: (2023)

DiaTool-DPO: Multi-Turn Direct Preference Optimization for Tool-Augmented Large Language Models
by: Jung, Sunghee, et al.
Published: (2025)

daDPO: Distribution-Aware DPO for Distilling Conversational Abilities
by: Zhang, Zhengze, et al.
Published: (2025)

Plan*RAG: Efficient Test-Time Planning for Retrieval Augmented Generation
by: Verma, Prakhar, et al.
Published: (2024)

It Takes Two: Your GRPO Is Secretly DPO
by: Wu, Yihong, et al.
Published: (2025)

SP^2DPO: An LLM-assisted Semantic Per-Pair DPO Generalization
by: He, Chaoyue, et al.
Published: (2026)

DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedback
by: Chen, Yangyi, et al.
Published: (2023)

CodeUltraFeedback: An LLM-as-a-Judge Dataset for Aligning Large Language Models to Coding Preferences
by: Weyssow, Martin, et al.
Published: (2024)

MixDPO: Modeling Preference Strength for Pluralistic Alignment
by: Imai, Saki, et al.
Published: (2026)

mDPO: Conditional Preference Optimization for Multimodal Large Language Models
by: Wang, Fei, et al.
Published: (2024)

Task Facet Learning: A Structured Approach to Prompt Optimization
by: Juneja, Gurusha, et al.
Published: (2024)

A Language Anchor-Guided Method for Robust Noisy Domain Generalization
by: Dai, Zilin, et al.
Published: (2025)

Understanding the Performance Gap in Preference Learning: A Dichotomy of RLHF and DPO
by: Shi, Ruizhe, et al.
Published: (2025)

Align to Structure: Aligning Large Language Models with Structural Information
by: Kim, Zae Myung, et al.
Published: (2025)

Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios?
by: Riabi, Arij, et al.
Published: (2021)

How Does DPO Reduce Toxicity? A Mechanistic Neuron-Level Analysis
by: Yang, Yushi, et al.
Published: (2024)

AdaDPO: Self-Adaptive Direct Preference Optimization with Balanced Gradient Updates
by: Chen, Shaolong, et al.
Published: (2026)

GIFT: Group-Relative Implicit Fine-Tuning Integrates GRPO with DPO and UNA
by: Wang, Zhichao
Published: (2025)

NoFunEval: Funny How Code LMs Falter on Requirements Beyond Functional Correctness
by: Singhal, Manav, et al.
Published: (2024)

From Noisy Traces to Stable Gradients: Bias-Variance Optimized Preference Optimization for Aligning Large Reasoning Models
by: Zhu, Mingkang, et al.
Published: (2025)

Provable Scaling Laws for the Test-Time Compute of Large Language Models
by: Chen, Yanxi, et al.
Published: (2024)

Make Some Noise: Unlocking Language Model Parallel Inference Capability through Noisy Training
by: Wang, Yixuan, et al.
Published: (2024)

Investigating Automatic Scoring and Feedback using Large Language Models
by: Katuka, Gloria Ashiya, et al.
Published: (2024)

On Provable Length and Compositional Generalization
by: Ahuja, Kartik, et al.
Published: (2024)

VRPO: Rethinking Value Modeling for Robust RL Training under Noisy Supervision
by: Zhu, Dingwei, et al.
Published: (2025)

Flow-DPO: Improving LLM Mathematical Reasoning through Online Multi-Agent Learning
by: Deng, Yihe, et al.
Published: (2024)