Imagen de Portada

Effect of Random Learning Rate: Theoretical Analysis of SGD Dynamics in Non-Convex Optimization via Stationary Distribution

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Yoshida, Naoki, Nakakita, Shogo, Imaizumi, Masaaki
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Benign Overfitting in Time Series Linear Models with Over-Parameterization
por: Nakakita, Shogo, et al.
Publicado: (2022)

Federated Learning with Relative Fairness
por: Nakakita, Shogo, et al.
Publicado: (2024)

Zero Generalization Error Theorem for Random Interpolators via Algebraic Geometry
por: Yoshida, Naoki, et al.
Publicado: (2025)

Optimal Dynamic Regret by Transformers for Non-Stationary Reinforcement Learning
por: Chen, Baiyuan, et al.
Publicado: (2025)

Non-asymptotic analysis of Langevin-type Monte Carlo algorithms
por: Nakakita, Shogo
Publicado: (2023)

Dimension-free Bounds for Sum of Dependent Matrices and Operators with Heavy-Tailed Distribution
por: Nakakita, Shogo, et al.
Publicado: (2022)

Sharp concentration of uniform generalization errors in binary linear classification
por: Nakakita, Shogo
Publicado: (2025)

On sample complexity for covariance estimation via the unadjusted Langevin algorithm
por: Nakakita, Shogo
Publicado: (2026)

Dimension-free uniform concentration bound for logistic regression
por: Nakakita, Shogo
Publicado: (2024)

Approximation of Permutation Invariant Polynomials by Transformers: Efficient Construction in Column-Size
por: Takeshita, Naoki, et al.
Publicado: (2025)

High-dimensional Nonparametric Contextual Bandit Problem
por: Iwazaki, Shogo, et al.
Publicado: (2025)

Effect of Weight Quantization on Learning Models by Typical Case Analysis
por: Kashiwamura, Shuhei, et al.
Publicado: (2024)

Extended Wasserstein-GAN Approach to Causal Distribution Learning: Density-Free Estimation and Minimax Optimality
por: Tamano, Shu, et al.
Publicado: (2026)

Precise Dynamics of Diagonal Linear Networks: A Unifying Analysis by Dynamical Mean-Field Theory
por: Nishiyama, Sota, et al.
Publicado: (2025)

Neuron Block Dynamics for XOR Classification with Zero-Margin
por: Braun, Guillaume, et al.
Publicado: (2026)

Bayesian Analysis for Over-parameterized Linear Model via Effective Spectra
por: Wakayama, Tomoya, et al.
Publicado: (2023)

High-Dimensional Limit of Stochastic Gradient Flow via Dynamical Mean-Field Theory
por: Nishiyama, Sota, et al.
Publicado: (2026)

Minimax Rates of Estimation for Optimal Transport Map between Infinite-Dimensional Spaces
por: Ponnoprat, Donlapark, et al.
Publicado: (2025)

Infinite-Width Limit of a Single Attention Layer: Analysis via Tensor Programs
por: Sakai, Mana, et al.
Publicado: (2025)

High-dimensional Contextual Bandit Problem without Sparsity
por: Komiyama, Junpei, et al.
Publicado: (2023)

Spectrum-Adaptive Generalization Bounds for Trained Deep Transformers
por: Sakai, Mana, et al.
Publicado: (2026)

Near-Optimal Algorithm for Non-Stationary Kernelized Bandits
por: Iwazaki, Shogo, et al.
Publicado: (2024)

Non-Euclidean SGD for Structured Optimization: Unified Analysis and Improved Rates
por: Kovalev, Dmitry, et al.
Publicado: (2025)

Automatic Domain Adaptation by Transformers in In-Context Learning
por: Hataya, Ryuichiro, et al.
Publicado: (2024)

Dichotomy of Feature Learning and Unlearning: Fast-Slow Analysis on Neural Networks with Stochastic Gradient Descent
por: Imai, Shota, et al.
Publicado: (2026)

Online Non-Stationary Stochastic Quasar-Convex Optimization
por: Pun, Yuen-Man, et al.
Publicado: (2024)

Improved Regret Analysis in Gaussian Process Bandits: Optimality for Noiseless Reward, RKHS norm, and Non-Stationary Variance
por: Iwazaki, Shogo, et al.
Publicado: (2025)

Fast Escape, Slow Convergence: Learning Dynamics of Phase Retrieval under Power-Law Data
por: Braun, Guillaume, et al.
Publicado: (2025)

Finite-Sample Inference for Sparsely Permuted Linear Regression
por: Ota, Hirofumi, et al.
Publicado: (2026)

Tight Long-Term Tail Decay of (Clipped) SGD in Non-Convex Optimization
por: Armacki, Aleksandar, et al.
Publicado: (2026)

Learning a Single Index Model from Anisotropic Data with vanilla Stochastic Gradient Descent
por: Braun, Guillaume, et al.
Publicado: (2025)

Anti Mode-Collapse in Mean-Field Transformer via Auxiliary Variables
por: Imaizumi, Masaaki, et al.
Publicado: (2026)

SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization
por: Dahan, Tehila, et al.
Publicado: (2023)

A Modular Algorithm for Non-Stationary Online Convex-Concave Optimization
por: Meng, Qing-xin, et al.
Publicado: (2025)

The Computational Complexity of Finding Stationary Points in Non-Convex Optimization
por: Hollender, Alexandros, et al.
Publicado: (2023)

Precise gradient descent training dynamics for finite-width multi-layer neural networks
por: Han, Qiyang, et al.
Publicado: (2025)

Iteration and Stochastic First-order Oracle Complexities of Stochastic Gradient Descent using Constant and Decaying Learning Rates
por: Imaizumi, Kento, et al.
Publicado: (2024)

Stochastic-Sign SGD for Federated Learning with Theoretical Guarantees
por: Jin, Richeng, et al.
Publicado: (2020)

Understanding Outer Optimizers in Local SGD: Learning Rates, Momentum, and Acceleration
por: Khaled, Ahmed, et al.
Publicado: (2025)

Risk-averse Learning with Non-Stationary Distributions
por: Wang, Siyi, et al.
Publicado: (2024)