Cover Image

Efficient Federated RLHF via Zeroth-Order Policy Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Deyi, Zhang, Qining, Ying, Lei
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Zeroth-Order Policy Gradient for Reinforcement Learning from Human Feedback without Reward Inference
by: Zhang, Qining, et al.
Published: (2024)

Provable Reinforcement Learning from Human Feedback with an Unknown Link Function
by: Zhang, Qining, et al.
Published: (2025)

Fast and Regret Optimal Best Arm Identification: Fundamental Limits and Low-Complexity Algorithms
by: Zhang, Qining, et al.
Published: (2023)

Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimization
by: Li, Zhe, et al.
Published: (2024)

Zeroth-Order Optimization is Secretly Single-Step Policy Optimization
by: Qiu, Junbin, et al.
Published: (2025)

Data-Free Black-Box Federated Learning via Zeroth-Order Gradient Estimation
by: Ma, Xinge, et al.
Published: (2025)

Reinforcement Learning from Human Feedback without Reward Inference: Model-Free Algorithm and Instance-Dependent Analysis
by: Zhang, Qining, et al.
Published: (2024)

Cost Aware Best Arm Identification
by: Kanarios, Kellen, et al.
Published: (2024)

Converge Faster, Talk Less: Hessian-Informed Federated Zeroth-Order Optimization
by: Li, Zhe, et al.
Published: (2025)

Learning Dynamics of Zeroth-Order Optimization: A Kernel Perspective
by: Li, Zhe, et al.
Published: (2026)

On the Convergence of Zeroth-Order Federated Tuning for Large Language Models
by: Ling, Zhenqing, et al.
Published: (2024)

Communication-Efficient and Differentially Private Vertical Federated Learning with Zeroth-Order Optimization
by: Zhang, Jianing, et al.
Published: (2025)

On Adaptivity in Zeroth-Order Optimization
by: Dbouk, Hassan, et al.
Published: (2026)

Fed-ZOE: Communication-Efficient Over-the-Air Federated Learning via Zeroth-Order Estimation
by: Jang, Jonggyu, et al.
Published: (2024)

When Foresight Pruning Meets Zeroth-Order Optimization: Efficient Federated Learning for Low-Memory Devices
by: Zhang, Pengyu, et al.
Published: (2024)

Revisiting Zeroth-Order Hessian Approximation: A Single-Step Policy Optimization Lens
by: Qiu, Junbin, et al.
Published: (2026)

Prior-Informed Zeroth-Order Optimization with Adaptive Direction Alignment for Memory-Efficient LLM Fine-Tuning
by: Jin, Feihu, et al.
Published: (2026)

Refining Adaptive Zeroth-Order Optimization at Ease
by: Shu, Yao, et al.
Published: (2025)

Private Zeroth-Order Optimization with Public Data
by: Gong, Xuchen, et al.
Published: (2025)

The Multi-Query Paradox in Zeroth-Order Optimization
by: Lin, Wei, et al.
Published: (2025)

Universally Empowering Zeroth-Order Optimization via Adaptive Layer-wise Sampling
by: Wang, Fei, et al.
Published: (2026)

Learning a Zeroth-Order Optimizer for Fine-Tuning LLMs
by: Zhang, Kairun, et al.
Published: (2025)

Position: Zeroth-Order Optimization in Deep Learning Is Underexplored, Not Underpowered
by: Liu, Sijia, et al.
Published: (2026)

Zeroth-Order Optimization at the Edge of Stability
by: Song, Minhak, et al.
Published: (2026)

Model Evolution Under Zeroth-Order Optimization: A Neural Tangent Kernel Perspective
by: Zhang, Chen, et al.
Published: (2026)

A Historical Trajectory Assisted Optimization Method for Zeroth-Order Federated Learning
by: Wu, Chenlin, et al.
Published: (2024)

Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark
by: Zhang, Yihua, et al.
Published: (2024)

Private Zeroth-Order Nonsmooth Nonconvex Optimization
by: Zhang, Qinzi, et al.
Published: (2024)

Robust and Efficient Zeroth-Order LLM Fine-Tuning via Adaptive Bayesian Subspace Optimizer
by: Feng, Jian, et al.
Published: (2026)

Powering Up Zeroth-Order Training via Subspace Gradient Orthogonalization
by: Lang, Yicheng, et al.
Published: (2026)

Zeroth-Order Optimization Finds Flat Minima
by: Zhang, Liang, et al.
Published: (2025)

On-Device Fine-Tuning via Backprop-Free Zeroth-Order Optimization
by: Katti, Prabodh, et al.
Published: (2025)

Policy Optimization in RLHF: The Impact of Out-of-preference Data
by: Li, Ziniu, et al.
Published: (2023)

ElasticZO: A Memory-Efficient On-Device Learning with Combined Zeroth- and First-Order Optimization
by: Sugiura, Keisuke, et al.
Published: (2025)

DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training
by: Chen, Aochuan, et al.
Published: (2023)

Compander-Aligned Query Geometry for Quantized Zeroth-Order Optimization
by: Shu, Yao, et al.
Published: (2026)

Model-Agnostic Zeroth-Order Policy Optimization for Meta-Learning of Ergodic Linear Quadratic Regulators
by: Pan, Yunian, et al.
Published: (2024)

Simultaneous Computation and Memory Efficient Zeroth-Order Optimizer for Fine-Tuning Large Language Models
by: Wang, Fei, et al.
Published: (2024)

Dataset Reset Policy Optimization for RLHF
by: Chang, Jonathan D., et al.
Published: (2024)

CurvZO: Adaptive Curvature-Guided Sparse Zeroth-Order Optimization for Efficient LLM Fine-Tuning
by: Wang, Shuo, et al.
Published: (2026)