Image de couverture de livre

MARS: Unleashing the Power of Variance Reduction for Training Large Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Yuan, Huizhuo, Liu, Yifeng, Wu, Shuang, Zhou, Xun, Gu, Quanquan
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Optimization and Control
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

MARS-M: When Variance Reduction Meets Matrices
par: Liu, Yifeng, et autres
Publié: (2025)

Towards Simple and Provable Parameter-Free Adaptive Gradient Methods
par: Tao, Yuanzhe, et autres
Publié: (2024)

Variance-Aware Feel-Good Thompson Sampling for Contextual Bandits
par: Li, Xuheng, et autres
Publié: (2025)

Variance-Aware Regret Bounds for Stochastic Contextual Dueling Bandits
par: Di, Qiwei, et autres
Publié: (2023)

Muon is Provably Faster with Momentum Variance Reduction
par: Qian, Xun, et autres
Publié: (2025)

Understanding SGD with Exponential Moving Average: A Case Study in Linear Regression
par: Li, Xuheng, et autres
Publié: (2025)

Unified Convergence Analysis for Score-Based Diffusion Models with Deterministic Samplers
par: Li, Runjia, et autres
Publié: (2024)

On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization
par: Zhou, Dongruo, et autres
Publié: (2018)

Accelerating RLHF Training with Reward Variance Increase
par: Yang, Zonglin, et autres
Publié: (2025)

Gradient Estimation and Variance Reduction in Stochastic and Deterministic Models
par: Keane, Ronan
Publié: (2024)

PPI-SVRG: Unifying Prediction-Powered Inference and Variance Reduction for Semi-Supervised Optimization
par: Ao, Ruicheng, et autres
Publié: (2026)

Feel-Good Thompson Sampling for Contextual Dueling Bandits
par: Li, Xuheng, et autres
Publié: (2024)

Optimal Horizon-Free Reward-Free Exploration for Linear Mixture MDPs
par: Zhang, Junkai, et autres
Publié: (2023)

A Nearly Optimal and Low-Switching Algorithm for Reinforcement Learning with General Function Approximation
par: Zhao, Heyang, et autres
Publié: (2023)

Stochastic Gradient Langevin Dynamics with Variance Reduction
par: Huang, Zhishen, et autres
Publié: (2021)

Provably Efficient Representation Selection in Low-rank Markov Decision Processes: From Online to Offline RL
par: Zhang, Weitong, et autres
Publié: (2021)

Pessimistic Nonlinear Least-Squares Value Iteration for Offline Reinforcement Learning
par: Di, Qiwei, et autres
Publié: (2023)

Dimension-Independent Convergence of Underdamped Langevin Monte Carlo in KL Divergence
par: Zhang, Shiyuan, et autres
Publié: (2026)

Adaptive Variance Reduction for Stochastic Optimization under Weaker Assumptions
par: Jiang, Wei, et autres
Publié: (2024)

TRSVR: An Adaptive Stochastic Trust-Region Method with Variance Reduction
par: Fang, Yuchen, et autres
Publié: (2026)

Matching the Statistical Query Lower Bound for $k$-Sparse Parity Problems with Sign Stochastic Gradient Descent
par: Kou, Yiwen, et autres
Publié: (2024)

Efficient Sign-Based Optimization: Accelerating Convergence via Variance Reduction
par: Jiang, Wei, et autres
Publié: (2024)

VAMO: Efficient Zeroth-Order Variance Reduction for SGD with Faster Convergence
par: Chen, Jiahe, et autres
Publié: (2025)

Policy-based Primal-Dual Methods for Concave CMDP with Variance Reduction
par: Ying, Donghao, et autres
Publié: (2022)

GradPower: Powering Gradients for Faster Language Model Pre-Training
par: Wang, Jinbo, et autres
Publié: (2025)

Projected Forward Gradient-Guided Frank-Wolfe Algorithm via Variance Reduction
par: Rostami, M., et autres
Publié: (2024)

Drago: Primal-Dual Coupled Variance Reduction for Faster Distributionally Robust Optimization
par: Mehta, Ronak, et autres
Publié: (2024)

Global Convergence of Natural Policy Gradient with Hessian-aided Momentum Variance Reduction
par: Feng, Jie, et autres
Publié: (2024)

Infeasible Deterministic, Stochastic, and Variance-Reduction Algorithms for Optimization under Orthogonality Constraints
par: Ablin, Pierre, et autres
Publié: (2023)

Variance Reduction and Low Sample Complexity in Stochastic Optimization via Proximal Point Method
par: Liang, Jiaming
Publié: (2024)

Projection-Free Variance Reduction Methods for Stochastic Constrained Multi-Level Compositional Optimization
par: Jiang, Wei, et autres
Publié: (2024)

Variance Reduction Methods Do Not Need to Compute Full Gradients: Improved Efficiency through Shuffling
par: Medyakov, Daniil, et autres
Publié: (2025)

Double Variance Reduction: A Smoothing Trick for Composite Optimization Problems without First-Order Gradient
par: Di, Hao, et autres
Publié: (2024)

Global Convergence and Rich Feature Learning in $L$-Layer Infinite-Width Neural Networks under $μ$P Parametrization
par: Chen, Zixiang, et autres
Publié: (2025)

Stochastic Variance-Reduced Newton: Accelerating Finite-Sum Minimization with Large Batches
par: Dereziński, Michał
Publié: (2022)

LoCo: Low-Bit Communication Adaptor for Large-scale Model Training
par: Xie, Xingyu, et autres
Publié: (2024)

Sample Complexity of Variance-reduced Distributionally Robust Q-learning
par: Wang, Shengbo, et autres
Publié: (2023)

GNMR: Runtime Stability Control for Low-Precision Large Language Model Training
par: Kong, Boao, et autres
Publié: (2026)

Reinforcement Learning from Human Feedback with Active Queries
par: Ji, Kaixuan, et autres
Publié: (2024)

Beyond Bounded Variance: Variance-Reduced Normalized Methods for Nonconvex Optimization under Blum-Gladyshev Noise
par: Upadhyay, Antesh, et autres
Publié: (2026)