Cover Image

KL-regularization Itself is Differentially Private in Bandits and RLHF

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Yizhou, Panaganti, Kishan, Shi, Laixi, Ziani, Juba, Wierman, Adam
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Efficient Policy Optimization in Robust Constrained MDPs with Iteration Complexity Guarantees
by: Ganguly, Sourav, et al.
Published: (2025)

Hybrid Transfer Reinforcement Learning: Provable Sample Efficiency from Shifted-Dynamics Data
by: Qu, Chengrui, et al.
Published: (2024)

Differentially Private Data Release on Graphs: Inefficiencies and Unfairness
by: Fioretto, Ferdinando, et al.
Published: (2024)

Distributionally Robust Constrained Reinforcement Learning under Strong Duality
by: Zhang, Zhengfei, et al.
Published: (2024)

Model-Free Robust $ϕ$-Divergence Reinforcement Learning Using Both Offline and Online Data
by: Panaganti, Kishan, et al.
Published: (2024)

Tractable Equilibrium Computation in Markov Games through Risk Aversion
by: Mazumdar, Eric, et al.
Published: (2024)

Offline and Online KL-Regularized RLHF under Differential Privacy
by: Wu, Yulian, et al.
Published: (2025)

Understanding Agent Scaling in LLM-Based Multi-Agent Systems via Diversity
by: Yang, Yingxuan, et al.
Published: (2026)

Robust Gymnasium: A Unified Modular Benchmark for Robust Reinforcement Learning
by: Gu, Shangding, et al.
Published: (2025)

Distributionally Robust Cooperative Multi-Agent Reinforcement Learning via Robust Value Factorization
by: Qu, Chengrui, et al.
Published: (2026)

Robust LLM Alignment via Distributionally Robust Direct Preference Optimization
by: Xu, Zaiyan, et al.
Published: (2025)

Group Distributionally Robust Optimization-Driven Reinforcement Learning for LLM Reasoning
by: Panaganti, Kishan, et al.
Published: (2026)

Generalisation of RLHF under Reward Shift and Clipped KL Regularisation
by: Tang, Kenton, et al.
Published: (2026)

Fusing Reward and Dueling Feedback in Stochastic Bandits
by: Wang, Xuchuang, et al.
Published: (2025)

Contextual Combinatorial Bandits with Probabilistically Triggered Arms
by: Liu, Xutong, et al.
Published: (2023)

Limits of Personalizing Differential Privacy Budgets
by: Cyffers, Edwige, et al.
Published: (2026)

Rethinking KL Regularization in RLHF: From Value Estimation to Gradient Optimization
by: Liu, Kezhao, et al.
Published: (2025)

Conceptual Belief-Informed Reinforcement Learning
by: Gu, Xingrui, et al.
Published: (2024)

Can LLMs Guide Their Own Exploration? Gradient-Guided Reinforcement Learning for LLM Reasoning
by: Liang, Zhenwen, et al.
Published: (2025)

Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-Constraint
by: Xiong, Wei, et al.
Published: (2023)

Guided Self-Evolving LLMs with Minimal Human Supervision
by: Yu, Wenhao, et al.
Published: (2025)

A Unified Theoretical Analysis of Private and Robust Offline Alignment: from RLHF to DPO
by: Zhou, Xingyu, et al.
Published: (2025)

Fairness Issues and Mitigations in (Differentially Private) Socio-Demographic Data Processes
by: Ko, Joonhyuk, et al.
Published: (2024)

When AI Eats Itself: On the Caveats of AI Autophagy
by: Xing, Xiaodan, et al.
Published: (2024)

Fairness-Regularized Online Optimization with Switching Costs
by: Li, Pengfei, et al.
Published: (2025)

Sample-Efficient Robust Multi-Agent Reinforcement Learning in the Face of Environmental Uncertainty
by: Shi, Laixi, et al.
Published: (2024)

Differential Privacy Overview and Fundamental Techniques
by: Fioretto, Ferdinando, et al.
Published: (2024)

ROCM: RLHF on consistency models
by: Shekhar, Shivanshu, et al.
Published: (2025)

Sharp Analysis for KL-Regularized Contextual Bandits and RLHF
by: Zhao, Heyang, et al.
Published: (2024)

Near-Optimal Regret for KL-Regularized Multi-Armed Bandits
by: Ji, Kaixuan, et al.
Published: (2026)

RLHF Workflow: From Reward Modeling to Online RLHF
by: Dong, Hanze, et al.
Published: (2024)

On the Optimal Sample Complexity of Offline Multi-Armed Bandits with KL Regularization
by: Ji, Kaixuan, et al.
Published: (2026)

AdamS: Momentum Itself Can Be A Normalizer for LLM Pretraining and Post-training
by: Zhang, Huishuai, et al.
Published: (2025)

Overcoming the Curse of Dimensionality in Reinforcement Learning Through Approximate Factorization
by: Lu, Chenbei, et al.
Published: (2024)

RLHF and IIA: Perverse Incentives
by: Xu, Wanqiao, et al.
Published: (2023)

Provable Differentially Private Computation of the Cross-Attention Mechanism
by: Ke, Yekun, et al.
Published: (2024)

Greedy Sampling Is Provably Efficient for RLHF
by: Wu, Di, et al.
Published: (2025)

A KL-regularization Framework for Learning to Plan with Adaptive Priors
by: Serra-Gomez, Álvaro, et al.
Published: (2025)

Boosting Deductive Reasoning with Step Signals In RLHF
by: Li, Jialian, et al.
Published: (2024)

The Hidden Link Between RLHF and Contrastive Learning
by: Lv, Xufei, et al.
Published: (2025)