Imagen de Portada

A theoretical and empirical study of new adaptive algorithms with additional momentum steps and shifted updates for stochastic non-convex optimization

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autor principal:	Alecsa, Cristian Daniel
Formato:	Preprint
Publicado:	2021
Materias:	Optimization and Control Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Convergence rates of stochastic gradient method with independent sequences of step-size and momentum weight
por: Hwang, Wen-Liang
Publicado: (2024)

Instance-optimal stochastic convex optimization: Can we improve upon sample-average and robust stochastic approximation?
por: Jiang, Liwei, et al.
Publicado: (2026)

Primal-dual algorithm for contextual stochastic combinatorial optimization
por: Bouvier, Louis, et al.
Publicado: (2025)

Online estimation of the inverse of the Hessian for stochastic optimization with application to universal stochastic Newton algorithms
por: Godichon-Baggioni, Antoine, et al.
Publicado: (2024)

Empirical and computer-aided robustness analysis of long-step and accelerated methods in smooth convex optimization
por: Vernimmen, Pierre, et al.
Publicado: (2025)

An accelerated first-order regularized momentum descent ascent algorithm for stochastic nonconvex-concave minimax problems
por: Zhang, Huiling, et al.
Publicado: (2023)

A framework for bilevel optimization that enables stochastic and global variance reduction algorithms
por: Dagréou, Mathieu, et al.
Publicado: (2022)

Local adapt-then-combine algorithms for distributed nonsmooth optimization: Achieving provable communication acceleration
por: Guo, Luyao, et al.
Publicado: (2026)

New logarithmic step size for stochastic gradient descent
por: Shamaee, M. Soheil, et al.
Publicado: (2024)

First-order methods for stochastic and finite-sum convex optimization with deterministic constraints
por: Lu, Zhaosong, et al.
Publicado: (2025)

Cubic regularized subspace Newton for non-convex optimization
por: Zhao, Jim, et al.
Publicado: (2024)

A block-coordinate descent framework for non-convex composite optimization. Application to sparse precision matrix estimation
por: Lauga, Guillaume
Publicado: (2026)

On the SAGA algorithm with decreasing step
por: Fredes, Luis, et al.
Publicado: (2024)

A simple uniformly optimal method without line search for convex optimization
por: Li, Tianjiao, et al.
Publicado: (2023)

Tradeoffs between convergence rate and noise amplification for momentum-based accelerated optimization algorithms
por: Mohammadi, Hesameddin, et al.
Publicado: (2022)

Non-geodesically-convex optimization in the Wasserstein space
por: Luu, Hoang Phuc Hau, et al.
Publicado: (2024)

Convergence of empirical subgradients for optimal transport-based objectives
por: Le, Tam
Publicado: (2026)

A simple and improved algorithm for noisy, convex, zeroth-order optimisation
por: Carpentier, Alexandra
Publicado: (2024)

A stochastic gradient method for trilevel optimization
por: Giovannelli, Tommaso, et al.
Publicado: (2025)

Projected gradient methods for nonconvex and stochastic smooth optimization: new complexities and auto-conditioned stepsizes
por: Lan, Guanghui, et al.
Publicado: (2024)

Learning based convex approximation for constrained parametric optimization
por: Liu, Kang, et al.
Publicado: (2025)

Reinforcement learning for adaptive interior point methods in convex quadratic programming
por: Bertoncini, Jeremy, et al.
Publicado: (2025)

Nesterov acceleration in benignly non-convex landscapes
por: Gupta, Kanan, et al.
Publicado: (2024)

A new use of the Kurdyka-Lojasiewicz property to study asymptotic behaviours of some stochastic optimization algorithms in a non-convex differentiable framework
por: Fest, Jean-Baptiste
Publicado: (2023)

Variance reduction techniques for stochastic proximal point algorithms
por: Traoré, Cheik, et al.
Publicado: (2023)

Sparse Polyak: an adaptive step size rule for high-dimensional M-estimation
por: Qiao, Tianqi, et al.
Publicado: (2025)

Glocal Smoothness: Line search and adaptive step sizes can help in theory too!
por: Fox, Curtis, et al.
Publicado: (2025)

Dealing with unbounded gradients in stochastic saddle-point optimization
por: Neu, Gergely, et al.
Publicado: (2024)

Fast sparse optimization via adaptive shrinkage
por: Cerone, Vito, et al.
Publicado: (2025)

A short proof of near-linear convergence of adaptive gradient descent under fourth-order growth and convexity
por: Davis, Damek, et al.
Publicado: (2026)

Multiscale replay: A robust algorithm for stochastic variational inequalities with a Markovian buffer
por: Nakul, Milind, et al.
Publicado: (2026)

A stochastic gradient descent algorithm with random search directions
por: Gbaguidi, Eméric
Publicado: (2025)

ICNN-enhanced 2SP: Leveraging input convex neural networks for solving two-stage stochastic programming
por: Liu, Yu, et al.
Publicado: (2025)

Unregularized limit of stochastic gradient method for Wasserstein distributionally robust optimization
por: Le, Tam
Publicado: (2025)

A learning-based approach to stochastic optimal control under reach-avoid constraint
por: Ni, Tingting, et al.
Publicado: (2024)

A new perspective on low-rank optimization
por: Bertsimas, Dimitris, et al.
Publicado: (2021)

Fast UCB-type algorithms for stochastic bandits with heavy and super heavy symmetric noise
por: Dorn, Yuriy, et al.
Publicado: (2024)

Langevin dynamics based algorithm e-TH$\varepsilon$O POULA for stochastic optimization problems with discontinuous stochastic gradient
por: Lim, Dong-Young, et al.
Publicado: (2022)

Fuzzy hyperparameters update in a second order optimization
por: Bensadok, Abdelaziz, et al.
Publicado: (2024)

Sinkhorn algorithms and linear programming solvers for optimal partial transport problems
por: Bai, Yikun
Publicado: (2024)