Imagen de Portada

Adapprox: Adaptive Approximation in Adam Optimization via Randomized Low-Rank Matrices

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Zhao, Pengxiang, Li, Ping, Gu, Yingjie, Zheng, Yi, Kölker, Stephan Ludger, Wang, Zhefeng, Yuan, Xiaoming
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Computation and Language Optimization and Control
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

A Convex-optimization-based Layer-wise Post-training Pruner for Large Language Models
por: Zhao, Pengxiang, et al.
Publicado: (2024)

GANQ: GPU-Adaptive Non-Uniform Quantization for Large Language Models
por: Zhao, Pengxiang, et al.
Publicado: (2025)

Riemannian Optimization for Hadamard Products of Low-Rank Matrices
por: Jawanpuria, Pratik, et al.
Publicado: (2026)

Clustering-based Low Rank Approximation Method
por: Zhu, Yujun, et al.
Publicado: (2025)

Multichannel Frequency Estimation with Constant Amplitude via Convex Structured Low-Rank Approximation
por: Wu, Xunmeng, et al.
Publicado: (2024)

Design of Transit Networks: Global Optimization of Continuous Approximation Models via Geometric Programming
por: Mao, Haoyang, et al.
Publicado: (2026)

Efficient Alternating Minimization with Applications to Weighted Low Rank Approximation
por: Song, Zhao, et al.
Publicado: (2023)

Factor Fitting, Rank Allocation, and Partitioning in Multilevel Low Rank Matrices
por: Parshakova, Tetiana, et al.
Publicado: (2023)

Understanding Adam Optimizer via Online Learning of Updates: Adam is FTRL in Disguise
por: Ahn, Kwangjun, et al.
Publicado: (2024)

Mitigate Overconservatism for Robust Optimization by Adapting to Opportunities
por: Lan, Yingjie
Publicado: (2021)

Fast and Accurate Estimation of Low-Rank Matrices from Noisy Measurements via Preconditioned Non-Convex Gradient Descent
por: Zhang, Gavin, et al.
Publicado: (2023)

Multi-Tubal Rank of Third Order Tensor and Related Low Rank Tensor Completion Problem
por: Yu, Quan, et al.
Publicado: (2020)

Low-Rank Extragradient Method for Nonsmooth and Low-Rank Matrix Optimization Problems
por: Garber, Dan, et al.
Publicado: (2022)

Low-Rank Mirror-Prox for Nonsmooth and Low-Rank Matrix Optimization Problems
por: Garber, Dan, et al.
Publicado: (2022)

MARS-M: When Variance Reduction Meets Matrices
por: Liu, Yifeng, et al.
Publicado: (2025)

Maximum Load Assortment Optimization: Approximation Algorithms and Adaptivity Gaps
por: Housni, Omar El, et al.
Publicado: (2023)

Dynamic Regret via Discounted-to-Dynamic Reduction with Applications to Curved Losses and Adam Optimizer
por: Xie, Yan-Feng, et al.
Publicado: (2026)

A Nearly Optimal and Low-Switching Algorithm for Reinforcement Learning with General Function Approximation
por: Zhao, Heyang, et al.
Publicado: (2023)

SPAP: Structured Pruning via Alternating Optimization and Penalty Methods
por: Hu, Hanyu, et al.
Publicado: (2025)

Compact Lifted Relaxations for Low-Rank Optimization
por: Cory-Wright, Ryan, et al.
Publicado: (2026)

Adam-SHANG: A Convergent Adam-Type Method for Stochastic Smooth Convex Optimization
por: Yu, Yaxin, et al.
Publicado: (2026)

From PowerSGD to PowerSGD+: Low-Rank Gradient Compression for Distributed Optimization with Convergence Guarantees
por: Xie, Shengping, et al.
Publicado: (2025)

On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization
por: Zhou, Dongruo, et al.
Publicado: (2018)

Provable Adaptivity of Adam under Non-uniform Smoothness
por: Wang, Bohan, et al.
Publicado: (2022)

Low-Rank Extragradient Methods for Scalable Semidefinite Optimization
por: Garber, Dan, et al.
Publicado: (2024)

Low Rank Matrix Completion via Robust Alternating Minimization in Nearly Linear Time
por: Gu, Yuzhou, et al.
Publicado: (2023)

Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance
por: Zhang, Qi, et al.
Publicado: (2024)

Multistage Robust Average Randomized Spectral Risk Optimization
por: Wu, Qiong, et al.
Publicado: (2024)

Approximation with Random Shallow ReLU Networks with Applications to Model Reference Adaptive Control
por: Lamperski, Andrew, et al.
Publicado: (2024)

Low-Tubal-Rank Tensor Recovery via Factorized Gradient Descent
por: Liu, Zhiyu, et al.
Publicado: (2024)

Polyhedral Analysis of Quadratic Optimization Problems with Stieltjes Matrices and Indicators
por: Liu, Peijing, et al.
Publicado: (2024)

Private and Robust Distributed Nonconvex Optimization via Polynomial Approximation
por: He, Zhiyu, et al.
Publicado: (2021)

Bernoulli-LoRA: A Theoretical Framework for Randomized Low-Rank Adaptation
por: Sokolov, Igor, et al.
Publicado: (2025)

Traffic Flow Data Completion and Anomaly Diagnosis via Sparse and Low-Rank Tensor Optimization
por: Man, Junxi, et al.
Publicado: (2025)

Convex Approximations of Random Constrained Markov Decision Processes
por: Varagapriya, V, et al.
Publicado: (2025)

LORENZA: Enhancing Generalization in Low-Rank Gradient LLM Training via Efficient Zeroth-Order Adaptive SAM
por: Refael, Yehonathan, et al.
Publicado: (2025)

Adaptive Clutter Suppression via Convex Optimization
por: He, Yifan, et al.
Publicado: (2025)

Reducing Contextual Stochastic Bilevel Optimization via Structured Function Approximation
por: Bouscary, Maxime, et al.
Publicado: (2025)

Improved Approximation Bounds for Moore-Penrose Inverses of Banded Matrices with Applications to Continuous-Time Linear Quadratic Control
por: Shin, Sungho, et al.
Publicado: (2024)

A Taylor-Bernstein Inner Approximation Algorithm for Path-Constrained Dynamic Optimization
por: Chang, Yuan, et al.
Publicado: (2026)