Cover Image

Square$χ$PO: Differentially Private and Robust $χ^2$-Preference Optimization in Offline Direct Alignment

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhou, Xingyu, Wu, Yulian, Weng, Wenqian, Orabona, Francesco
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Unified Theoretical Analysis of Private and Robust Offline Alignment: from RLHF to DPO
by: Zhou, Xingyu, et al.
Published: (2025)

Improved Bounds for Private and Robust Alignment
by: Weng, Wenqian, et al.
Published: (2025)

Offline and Online KL-Regularized RLHF under Differential Privacy
by: Wu, Yulian, et al.
Published: (2025)

Optimal Regret of Bernoulli Bandits under Global Differential Privacy
by: Azize, Achraf, et al.
Published: (2025)

Better-than-KL PAC-Bayes Bounds
by: Kuzborskij, Ilja, et al.
Published: (2024)

On the Sample Complexity of Differentially Private Policy Optimization
by: He, Yi, et al.
Published: (2025)

Forward $χ^2$ Divergence Based Variational Importance Sampling
by: Li, Chengrui, et al.
Published: (2023)

From Betting to Empirical Bernstein LIL
by: Orabona, Francesco
Published: (2026)

A Modern Introduction to Online Learning
by: Orabona, Francesco
Published: (2019)

A Note on How to Remove the $\ln\ln T$ Term from the Squint Bound
by: Orabona, Francesco
Published: (2026)

Robust LLM Alignment via Distributionally Robust Direct Preference Optimization
by: Xu, Zaiyan, et al.
Published: (2025)

STaR-Bets: Sequential Target-Recalculating Bets for Tighter Confidence Intervals
by: Voráček, Václav, et al.
Published: (2025)

Estimation of discrete distributions with high probability under $χ^2$-divergence
by: Louati, Sirine
Published: (2025)

CompassDPO: Dynamics-Controlled Direct Preference Optimization for Robust Safety Alignment
by: Liu, Jilong, et al.
Published: (2026)

Generalized Preference Optimization: A Unified Approach to Offline Alignment
by: Tang, Yunhao, et al.
Published: (2024)

Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimization
by: Wu, Junkang, et al.
Published: (2024)

Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex Conversion
by: Cutkosky, Ashok, et al.
Published: (2023)

Towards Differentially Private Reinforcement Learning with General Function Approximation
by: He, Yi, et al.
Published: (2026)

An Equivalence Between Static and Dynamic Regret Minimization
by: Jacobsen, Andrew, et al.
Published: (2024)

Self-Directed Learning of Convex Labelings on Graphs
by: Sokolov, Georgy, et al.
Published: (2024)

Hybrid Preference Optimization for Alignment: Provably Faster Convergence Rates by Combining Offline Preferences with Online Exploration
by: Bose, Avinandan, et al.
Published: (2024)

Lightweight Robust Direct Preference Optimization
by: Kim, Cheol Woo, et al.
Published: (2025)

Correcting the Mythos of KL-Regularization: Direct Alignment without Overoptimization via Chi-Squared Preference Optimization
by: Huang, Audrey, et al.
Published: (2024)

Differentially Private Preference Data Synthesis for Large Language Model Alignment
by: Gao, Fengyu, et al.
Published: (2026)

Differentially Private Non-convex Distributionally Robust Optimization
by: Xu, Difei, et al.
Published: (2026)

Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization
by: Zhou, Zhanhui, et al.
Published: (2023)

FairPO: Robust Preference Optimization for Fair Multi-Label Learning
by: Mondal, Soumen Kumar, et al.
Published: (2025)

When Determinants Are Not Enough: Private Rare Switching
by: Zhou, Xingyu
Published: (2026)

$(ε, δ)$-Differentially Private Partial Least Squares Regression
by: Nikzad-Langerodi, Ramin, et al.
Published: (2024)

New Perspectives on the Polyak Stepsize: Surrogate Functions and Negative Results
by: Orabona, Francesco, et al.
Published: (2025)

A Best-of-Both-Worlds Proof for Tsallis-INF without Fenchel Conjugates
by: Lee, Wei-Cheng, et al.
Published: (2025)

A Finite-Time Analysis of TD Learning with Linear Function Approximation without Projections or Strong Convexity
by: Lee, Wei-Cheng, et al.
Published: (2025)

Online Conformal Prediction via Universal Portfolio Algorithms
by: Liu, Tuo, et al.
Published: (2026)

New Lower Bounds for Stochastic Non-Convex Optimization through Divergence Decomposition
by: Saad, El Mehdi, et al.
Published: (2025)

Improving Group Robustness on Spurious Correlation via Evidential Alignment
by: Ye, Wenqian, et al.
Published: (2025)

Differentially Private Linear Bandits with Partial Distributed Feedback
by: Li, Fengjiao, et al.
Published: (2022)

Length Desensitization in Direct Preference Optimization
by: Liu, Wei, et al.
Published: (2024)

Randomized Least Squares Value Iteration itself is Joint Differentially Private
by: Lu, Haiyang, et al.
Published: (2026)

$χ$SPN: Characteristic Interventional Sum-Product Networks for Causal Inference in Hybrid Domains
by: Poonia, Harsh, et al.
Published: (2024)

Private Wasserstein Distance
by: Li, Wenqian, et al.
Published: (2024)