Cover Image

Understanding Sampler Stochasticity in Training Diffusion Models for RLHF

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Sheng, Jiayuan, Zhao, Hanyang, Chen, Haoxian, Yao, David D., Tang, Wenpin
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Score as Action: Fine-Tuning Diffusion Generative Models by Continuous-time Reinforcement Learning
by: Zhao, Hanyang, et al.
Published: (2025)

ART for Diffusion Sampling: A Reinforcement Learning Approach to Timestep Schedule
by: Huang, Yilie, et al.
Published: (2026)

Accelerating RLHF Training with Reward Variance Increase
by: Yang, Zonglin, et al.
Published: (2025)

MallowsPO: Fine-Tune Your LLM with Preference Dispersions
by: Chen, Haoxian, et al.
Published: (2024)

Scores as Actions: a framework of fine-tuning diffusion models by continuous-time reinforcement learning
by: Zhao, Hanyang, et al.
Published: (2024)

DiFFPO: Training Diffusion LLMs to Reason Fast and Furious via Reinforcement Learning
by: Zhao, Hanyang, et al.
Published: (2025)

Through the River: Understanding the Benefit of Schedule-Free Methods for Language Model Training
by: Song, Minhak, et al.
Published: (2025)

OPD+: Rethinking the Advantage Design for On-Policy Distillation
by: Zhao, Hanyang, et al.
Published: (2026)

StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models
by: Yu, Dingzhi, et al.
Published: (2026)

Unsupervised Training of Diffusion Models for Feasible Solution Generation in Neural Combinatorial Optimization
by: Hong, Seong-Hyun, et al.
Published: (2024)

Sampler-Robust Optimization under Generative Models
by: Zhang, Ziwei, et al.
Published: (2026)

Enhancing Stochastic Gradient Descent: A Unified Framework and Novel Acceleration Methods for Faster Convergence
by: Deng, Yichuan, et al.
Published: (2024)

Fine-tuning of diffusion models via stochastic control: entropy regularization and beyond
by: Tang, Wenpin, et al.
Published: (2024)

Training Safe Neural Networks with Global SDP Bounds
by: Soletskyi, Roman, et al.
Published: (2024)

MDNS: Masked Diffusion Neural Sampler via Stochastic Optimal Control
by: Zhu, Yuchen, et al.
Published: (2025)

Understanding Optimization in Deep Learning with Central Flows
by: Cohen, Jeremy M., et al.
Published: (2024)

Provable Acceleration for Diffusion Models under Minimal Assumptions
by: Li, Gen, et al.
Published: (2024)

RPO: Fine-Tuning Visual Generative Models via Rich Vision-Language Preferences
by: Zhao, Hanyang, et al.
Published: (2025)

Isotropic Curvature Model for Understanding Deep Learning Optimization: Is Gradient Orthogonalization Optimal?
by: Su, Weijie
Published: (2025)

The Sharpness Disparity Principle in Transformers for Accelerating Language Model Pre-Training
by: Wang, Jinbo, et al.
Published: (2025)

A Minimalist Bayesian Framework for Stochastic Optimization
by: Wang, Kaizheng
Published: (2025)

SGD at the Edge of Stability: The Stochastic Sharpness Gap
by: Liao, Fangshuo, et al.
Published: (2026)

Reward-Directed Score-Based Diffusion Models via q-Learning
by: Gao, Xuefeng, et al.
Published: (2024)

Stochastic Optimization of Inventory at Large-scale Supply Chains
by: Jin, Zhaoyang Larry, et al.
Published: (2025)

Almost Bayesian: The Fractal Dynamics of Stochastic Gradient Descent
by: Hennick, Max, et al.
Published: (2025)

Towards Faster Decentralized Stochastic Optimization with Communication Compression
by: Islamov, Rustem, et al.
Published: (2024)

Beyond Expectations: Learning with Stochastic Dominance Made Practical
by: Cen, Shicong, et al.
Published: (2024)

On the Convergence of (Stochastic) Gradient Descent for Kolmogorov--Arnold Networks
by: Gao, Yihang, et al.
Published: (2024)

A Retention-Centric Framework for Continual Learning with Guaranteed Model Developmental Safety
by: Li, Gang, et al.
Published: (2024)

Weighted Low-rank Approximation via Stochastic Gradient Descent on Manifolds
by: Xu, Conglong, et al.
Published: (2025)

Stochastic Subgradient Methods with Guaranteed Global Stability in Nonsmooth Nonconvex Optimization
by: Xiao, Nachuan, et al.
Published: (2023)

Neural Combinatorial Optimization for Stochastic Flexible Job Shop Scheduling Problems
by: Smit, Igor G., et al.
Published: (2024)

Asymptotic and Finite Sample Analysis of Nonexpansive Stochastic Approximations with Markovian Noise
by: Blaser, Ethan, et al.
Published: (2024)

Stochastic Optimization with Constraints: A Non-asymptotic Instance-Dependent Analysis
by: Khamaru, Koulik
Published: (2024)

Understanding Fixed Predictions via Confined Regions
by: Lawless, Connor, et al.
Published: (2025)

Training-free Diffusion Model Alignment with Sampling Demons
by: Yeh, Po-Hung, et al.
Published: (2024)

Training Infinitely Deep and Wide Transformers
by: Barboni, Raphaël, et al.
Published: (2026)

Anytime Training with Schedule-Free Spectral Optimization
by: Apte, Anuj, et al.
Published: (2026)

Learning to Specialize: Joint Gating-Expert Training for Adaptive MoEs in Decentralized Settings
by: Farhat, Yehya, et al.
Published: (2023)

Optimal and Diffusion Transports in Machine Learning
by: Peyré, Gabriel
Published: (2025)