Copertina

Adapprox: Adaptive Approximation in Adam Optimization via Randomized Low-Rank Matrices

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhao, Pengxiang, Li, Ping, Gu, Yingjie, Zheng, Yi, Kölker, Stephan Ludger, Wang, Zhefeng, Yuan, Xiaoming
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning Computation and Language Optimization and Control
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

A Convex-optimization-based Layer-wise Post-training Pruner for Large Language Models
di: Zhao, Pengxiang, et al.
Pubblicazione: (2024)

GANQ: GPU-Adaptive Non-Uniform Quantization for Large Language Models
di: Zhao, Pengxiang, et al.
Pubblicazione: (2025)

Riemannian Optimization for Hadamard Products of Low-Rank Matrices
di: Jawanpuria, Pratik, et al.
Pubblicazione: (2026)

Clustering-based Low Rank Approximation Method
di: Zhu, Yujun, et al.
Pubblicazione: (2025)

Multichannel Frequency Estimation with Constant Amplitude via Convex Structured Low-Rank Approximation
di: Wu, Xunmeng, et al.
Pubblicazione: (2024)

Design of Transit Networks: Global Optimization of Continuous Approximation Models via Geometric Programming
di: Mao, Haoyang, et al.
Pubblicazione: (2026)

Efficient Alternating Minimization with Applications to Weighted Low Rank Approximation
di: Song, Zhao, et al.
Pubblicazione: (2023)

Factor Fitting, Rank Allocation, and Partitioning in Multilevel Low Rank Matrices
di: Parshakova, Tetiana, et al.
Pubblicazione: (2023)

Understanding Adam Optimizer via Online Learning of Updates: Adam is FTRL in Disguise
di: Ahn, Kwangjun, et al.
Pubblicazione: (2024)

Mitigate Overconservatism for Robust Optimization by Adapting to Opportunities
di: Lan, Yingjie
Pubblicazione: (2021)

Fast and Accurate Estimation of Low-Rank Matrices from Noisy Measurements via Preconditioned Non-Convex Gradient Descent
di: Zhang, Gavin, et al.
Pubblicazione: (2023)

Multi-Tubal Rank of Third Order Tensor and Related Low Rank Tensor Completion Problem
di: Yu, Quan, et al.
Pubblicazione: (2020)

Low-Rank Extragradient Method for Nonsmooth and Low-Rank Matrix Optimization Problems
di: Garber, Dan, et al.
Pubblicazione: (2022)

Low-Rank Mirror-Prox for Nonsmooth and Low-Rank Matrix Optimization Problems
di: Garber, Dan, et al.
Pubblicazione: (2022)

MARS-M: When Variance Reduction Meets Matrices
di: Liu, Yifeng, et al.
Pubblicazione: (2025)

Maximum Load Assortment Optimization: Approximation Algorithms and Adaptivity Gaps
di: Housni, Omar El, et al.
Pubblicazione: (2023)

Dynamic Regret via Discounted-to-Dynamic Reduction with Applications to Curved Losses and Adam Optimizer
di: Xie, Yan-Feng, et al.
Pubblicazione: (2026)

A Nearly Optimal and Low-Switching Algorithm for Reinforcement Learning with General Function Approximation
di: Zhao, Heyang, et al.
Pubblicazione: (2023)

SPAP: Structured Pruning via Alternating Optimization and Penalty Methods
di: Hu, Hanyu, et al.
Pubblicazione: (2025)

Compact Lifted Relaxations for Low-Rank Optimization
di: Cory-Wright, Ryan, et al.
Pubblicazione: (2026)

Adam-SHANG: A Convergent Adam-Type Method for Stochastic Smooth Convex Optimization
di: Yu, Yaxin, et al.
Pubblicazione: (2026)

From PowerSGD to PowerSGD+: Low-Rank Gradient Compression for Distributed Optimization with Convergence Guarantees
di: Xie, Shengping, et al.
Pubblicazione: (2025)

On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization
di: Zhou, Dongruo, et al.
Pubblicazione: (2018)

Provable Adaptivity of Adam under Non-uniform Smoothness
di: Wang, Bohan, et al.
Pubblicazione: (2022)

Low-Rank Extragradient Methods for Scalable Semidefinite Optimization
di: Garber, Dan, et al.
Pubblicazione: (2024)

Low Rank Matrix Completion via Robust Alternating Minimization in Nearly Linear Time
di: Gu, Yuzhou, et al.
Pubblicazione: (2023)

Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance
di: Zhang, Qi, et al.
Pubblicazione: (2024)

Multistage Robust Average Randomized Spectral Risk Optimization
di: Wu, Qiong, et al.
Pubblicazione: (2024)

Approximation with Random Shallow ReLU Networks with Applications to Model Reference Adaptive Control
di: Lamperski, Andrew, et al.
Pubblicazione: (2024)

Low-Tubal-Rank Tensor Recovery via Factorized Gradient Descent
di: Liu, Zhiyu, et al.
Pubblicazione: (2024)

Polyhedral Analysis of Quadratic Optimization Problems with Stieltjes Matrices and Indicators
di: Liu, Peijing, et al.
Pubblicazione: (2024)

Private and Robust Distributed Nonconvex Optimization via Polynomial Approximation
di: He, Zhiyu, et al.
Pubblicazione: (2021)

Bernoulli-LoRA: A Theoretical Framework for Randomized Low-Rank Adaptation
di: Sokolov, Igor, et al.
Pubblicazione: (2025)

Traffic Flow Data Completion and Anomaly Diagnosis via Sparse and Low-Rank Tensor Optimization
di: Man, Junxi, et al.
Pubblicazione: (2025)

Convex Approximations of Random Constrained Markov Decision Processes
di: Varagapriya, V, et al.
Pubblicazione: (2025)

LORENZA: Enhancing Generalization in Low-Rank Gradient LLM Training via Efficient Zeroth-Order Adaptive SAM
di: Refael, Yehonathan, et al.
Pubblicazione: (2025)

Adaptive Clutter Suppression via Convex Optimization
di: He, Yifan, et al.
Pubblicazione: (2025)

Reducing Contextual Stochastic Bilevel Optimization via Structured Function Approximation
di: Bouscary, Maxime, et al.
Pubblicazione: (2025)

Improved Approximation Bounds for Moore-Penrose Inverses of Banded Matrices with Applications to Continuous-Time Linear Quadratic Control
di: Shin, Sungho, et al.
Pubblicazione: (2024)

A Taylor-Bernstein Inner Approximation Algorithm for Path-Constrained Dynamic Optimization
di: Chang, Yuan, et al.
Pubblicazione: (2026)