Cover Image

SketchySGD: Reliable Stochastic Optimization via Randomized Curvature Estimates

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Frangella, Zachary, Rathore, Pratik, Zhao, Shipu, Udell, Madeleine
Format:	Preprint
Published:	2022
Subjects:	Optimization and Control Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

PROMISE: Preconditioned Stochastic Optimization Methods by Incorporating Scalable Curvature Estimates
by: Frangella, Zachary, et al.
Published: (2023)

Challenges in Training PINNs: A Loss Landscape Perspective
by: Rathore, Pratik, et al.
Published: (2024)

Turbocharging Gaussian Process Inference with Approximate Sketch-and-Project
by: Rathore, Pratik, et al.
Published: (2025)

Have ASkotch: A Neat Solution for Large-scale Kernel Ridge Regression
by: Rathore, Pratik, et al.
Published: (2024)

GeNIOS: an (almost) second-order operator-splitting solver for large-scale convex optimization
by: Diamandis, Theo, et al.
Published: (2023)

An automatic system to detect equivalence between iterative algorithms
by: Zhao, Shipu, et al.
Published: (2021)

Small Gradient Norm Regret for Online Convex Optimization
by: Gao, Wenzhi, et al.
Published: (2026)

On the (linear) convergence of Generalized Newton Inexact ADMM
by: Frangella, Zachary, et al.
Published: (2023)

CRONOS: Enhancing Deep Learning with Scalable GPU Accelerated Convex Neural Networks
by: Feng, Miria, et al.
Published: (2024)

SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization
by: Dahan, Tehila, et al.
Published: (2023)

On the Provable Suboptimality of Momentum SGD in Nonstationary Stochastic Optimization
by: Sahu, Sharan, et al.
Published: (2026)

SAPPHIRE: Preconditioned Stochastic Variance Reduction for Faster Large-Scale Statistical Learning
by: Sun, Jingruo, et al.
Published: (2025)

$μ^2$-SGD: Stable Stochastic Optimization via a Double Momentum Mechanism
by: Dahan, Tehila, et al.
Published: (2023)

Gradient Methods with Online Scaling
by: Gao, Wenzhi, et al.
Published: (2024)

Gradient Methods with Online Scaling Part I. Theoretical Foundations
by: Gao, Wenzhi, et al.
Published: (2025)

Provable and Practical Online Learning Rate Adaptation with Hypergradient Descent
by: Chu, Ya-Chi, et al.
Published: (2025)

Gradient Methods with Online Scaling Part II. Practical Aspects
by: Chu, Ya-Chi, et al.
Published: (2025)

Understanding Fixed Predictions via Confined Regions
by: Lawless, Connor, et al.
Published: (2025)

Demystifying SGD with Doubly Stochastic Gradients
by: Kim, Kyurae, et al.
Published: (2024)

Sign-SGD via Parameter-Free Optimization
by: Medyakov, Daniil, et al.
Published: (2025)

Enhancing Stochastic Optimization for Statistical Efficiency Using ROOT-SGD with Diminishing Stepsize
by: Li, Chris Junchi
Published: (2024)

Edge of Stochastic Stability: Revisiting the Edge of Stability for SGD
by: Andreyev, Arseniy, et al.
Published: (2024)

Stochastic Optimization with Random Search
by: Chayti, El Mahdi, et al.
Published: (2025)

A Hessian-Aware Stochastic Differential Equation for Modelling SGD
by: Li, Xiang, et al.
Published: (2024)

Provable Complexity Improvement of AdaGrad over SGD: Upper and Lower Bounds in Stochastic Non-Convex Optimization
by: Jiang, Ruichen, et al.
Published: (2024)

From PowerSGD to PowerSGD+: Low-Rank Gradient Compression for Distributed Optimization with Convergence Guarantees
by: Xie, Shengping, et al.
Published: (2025)

Enhancing Physics-Informed Neural Networks Through Feature Engineering
by: Fazliani, Shaghayegh, et al.
Published: (2025)

The Optimality of (Accelerated) SGD for High-Dimensional Quadratic Optimization
by: Zhang, Haihan, et al.
Published: (2024)

SGD with Partial Hessian for Deep Neural Networks Optimization
by: Sun, Ying, et al.
Published: (2024)

Optimal Projection-Free Adaptive SGD for Matrix Optimization
by: Kovalev, Dmitry
Published: (2026)

SGD at the Edge of Stability: The Stochastic Sharpness Gap
by: Liao, Fangshuo, et al.
Published: (2026)

AutoSGD: Automatic Learning Rate Selection for Stochastic Gradient Descent
by: Surjanovic, Nikola, et al.
Published: (2025)

StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models
by: Yu, Dingzhi, et al.
Published: (2026)

Accelerating Distributed Stochastic Optimization via Self-Repellent Random Walks
by: Hu, Jie, et al.
Published: (2024)

Understanding Outer Optimizers in Local SGD: Learning Rates, Momentum, and Acceleration
by: Khaled, Ahmed, et al.
Published: (2025)

Non-Euclidean SGD for Structured Optimization: Unified Analysis and Improved Rates
by: Kovalev, Dmitry, et al.
Published: (2025)

Proactive DP: A Multple Target Optimization Framework for DP-SGD
by: van Dijk, Marten, et al.
Published: (2021)

Tight Long-Term Tail Decay of (Clipped) SGD in Non-Convex Optimization
by: Armacki, Aleksandar, et al.
Published: (2026)

Leveraging Coordinate Momentum in SignSGD and Muon: Memory-Optimized Zero-Order
by: Petrov, Egor, et al.
Published: (2025)

Making SGD Parameter-Free
by: Carmon, Yair, et al.
Published: (2022)