Copertina

Provably Convergent Primal-Dual DPO for Constrained LLM Alignment

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Du, Yihan, Kong, Seo Taek, Srikant, R.
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Finite-Sample Wasserstein Error Bounds and Concentration Inequalities for Nonlinear Stochastic Approximation
di: Kong, Seo Taek, et al.
Pubblicazione: (2026)

Noise Schedule Design for Diffusion Models: An Optimal Control Perspective
di: Kong, Seo Taek, et al.
Pubblicazione: (2026)

Provable Last-Iterate Convergence for Multi-Objective Safe LLM Alignment via Optimistic Primal-Dual
di: Li, Yining, et al.
Pubblicazione: (2026)

Nonasymptotic CLT and Error Bounds for Two-Time-Scale Stochastic Approximation
di: Kong, Seo Taek, et al.
Pubblicazione: (2025)

Spectral Clustering for Crowdsourcing with Inherently Distinct Task Types
di: Mandal, Saptarshi, et al.
Pubblicazione: (2023)

Cascading Reinforcement Learning
di: Du, Yihan, et al.
Pubblicazione: (2024)

Global Convergence for Average Reward Constrained MDPs with Primal-Dual Actor Critic Algorithm
di: Xu, Yang, et al.
Pubblicazione: (2025)

Last-Iterate Convergent Policy Gradient Primal-Dual Methods for Constrained MDPs
di: Ding, Dongsheng, et al.
Pubblicazione: (2023)

Conditional Equivalence of DPO and RLHF: Implicit Assumption, Failure Modes, and Provable Alignment
di: Yang, Zhiqin, et al.
Pubblicazione: (2026)

Rates of Convergence in the Central Limit Theorem for Markov Chains, with an Application to TD Learning
di: Srikant, R.
Pubblicazione: (2024)

Provably Mitigating Corruption, Overoptimization, and Verbosity Simultaneously in Offline and Online RLHF/DPO Alignment
di: Chen, Ziyi, et al.
Pubblicazione: (2025)

Reinforcement Learning with Segment Feedback
di: Du, Yihan, et al.
Pubblicazione: (2025)

Constrained Sampling with Primal-Dual Langevin Monte Carlo
di: Chamon, Luiz F. O., et al.
Pubblicazione: (2024)

Provable Robustness against Backdoor Attacks via the Primal-Dual Perspective on Differential Privacy
di: Saxena, Aman, et al.
Pubblicazione: (2026)

Hybrid Preference Optimization for Alignment: Provably Faster Convergence Rates by Combining Offline Preferences with Online Exploration
di: Bose, Avinandan, et al.
Pubblicazione: (2024)

A Primal-Dual Algorithm for Offline Constrained Reinforcement Learning with Linear MDPs
di: Hong, Kihyuk, et al.
Pubblicazione: (2024)

Convergence of Natural Policy Gradient for a Family of Infinite-State Queueing MDPs
di: Grosof, Isaac, et al.
Pubblicazione: (2024)

Provably Robust DPO: Aligning Language Models with Noisy Feedback
di: Chowdhury, Sayak Ray, et al.
Pubblicazione: (2024)

Exploration-Driven Policy Optimization in RLHF: Theoretical Insights on Efficient Data Utilization
di: Du, Yihan, et al.
Pubblicazione: (2024)

Linear Convergence of Entropy-Regularized Natural Policy Gradient with Linear Function Approximation
di: Cayci, Semih, et al.
Pubblicazione: (2021)

A Policy Gradient Primal-Dual Algorithm for Constrained MDPs with Uniform PAC Guarantees
di: Kitamura, Toshinori, et al.
Pubblicazione: (2024)

Convergence of Distributionally Robust Q-Learning with Linear Function Approximation
di: Mandal, Saptarshi, et al.
Pubblicazione: (2025)

Nearly Optimal Linear Convergence of Stochastic Primal-Dual Methods for Linear Programming
di: Lu, Haihao, et al.
Pubblicazione: (2021)

MGDA-Decoupled: Geometry-Aware Multi-Objective Optimisation for DPO-based LLM Alignment
di: Vári-Kakas, Andor, et al.
Pubblicazione: (2026)

Double Duality: Variational Primal-Dual Policy Optimization for Constrained Reinforcement Learning
di: Li, Zihao, et al.
Pubblicazione: (2024)

PAPAL: A Provable PArticle-based Primal-Dual ALgorithm for Mixed Nash Equilibrium
di: Ding, Shihong, et al.
Pubblicazione: (2023)

A Single-Loop Deep Actor-Critic Algorithm for Constrained Reinforcement Learning with Provable Convergence
di: Wang, Kexuan, et al.
Pubblicazione: (2023)

On the Convergence of Modified Policy Iteration in Risk Sensitive Exponential Cost Markov Decision Processes
di: Murthy, Yashaswini, et al.
Pubblicazione: (2023)

Primal-Dual Neural Algorithmic Reasoning
di: He, Yu, et al.
Pubblicazione: (2025)

Online Inference of Constrained Optimization: Primal-Dual Optimality and Sequential Quadratic Programming
di: Gao, Yihang, et al.
Pubblicazione: (2025)

Constrained Entropic Unlearning: A Primal-Dual Framework for Large Language Models
di: Entesari, Taha, et al.
Pubblicazione: (2025)

Continuous Learned Primal Dual
di: Runkel, Christina, et al.
Pubblicazione: (2024)

Robust Multi-Objective Preference Alignment with Online DPO
di: Gupta, Raghav, et al.
Pubblicazione: (2025)

SPARKLE: A Unified Single-Loop Primal-Dual Framework for Decentralized Bilevel Optimization
di: Zhu, Shuchen, et al.
Pubblicazione: (2024)

The Viscosity of Logic: Phase Transitions and Hysteresis in DPO Alignment
di: Pollanen, Marco
Pubblicazione: (2026)

C2-DPO: Constrained Controlled Direct Preference Optimization
di: Asadi, Kavosh, et al.
Pubblicazione: (2025)

Off-Policy Primal-Dual Safe Reinforcement Learning
di: Wu, Zifan, et al.
Pubblicazione: (2024)

A Primal-Dual Algorithm for Hybrid Federated Learning
di: Overman, Tom, et al.
Pubblicazione: (2022)

Weakly Convex Regularisers for Inverse Problems: Convergence of Critical Points and Primal-Dual Optimisation
di: Shumaylov, Zakhar, et al.
Pubblicazione: (2024)

SP^2DPO: An LLM-assisted Semantic Per-Pair DPO Generalization
di: He, Chaoyue, et al.
Pubblicazione: (2026)