Copertina

MAC: An Efficient Gradient Preconditioning using Mean Activation Approximated Curvature

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Seung, Hyunseok, Lee, Jaewoo, Ko, Hyunsuk
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

NysAct: A Scalable Preconditioned Gradient Descent using Nystrom Approximation
di: Seung, Hyunseok, et al.
Pubblicazione: (2025)

Low-Rank Curvature for Zeroth-Order Optimization in LLM Fine-Tuning
di: Seung, Hyunseok, et al.
Pubblicazione: (2025)

An Adaptive Method Stabilizing Activations for Enhanced Generalization
di: Seung, Hyunseok, et al.
Pubblicazione: (2025)

Sign Gradient Descent-based Neuronal Dynamics: ANN-to-SNN Conversion Beyond ReLU Network
di: Oh, Hyunseok, et al.
Pubblicazione: (2024)

Gradient Preconditioning for Efficient and Reliable Reward-Guided Generation
di: Hwang, Jisung, et al.
Pubblicazione: (2026)

On the Nystrom Approximation for Preconditioning in Kernel Machines
di: Abedsoltan, Amirhesam, et al.
Pubblicazione: (2023)

Preconditioned DeltaNet: Curvature-aware Sequence Modeling for Linear Recurrences
di: Tumma, Neehal, et al.
Pubblicazione: (2026)

Stochastic Gradient Methods with Preconditioned Updates
di: Sadiev, Abdurakhmon, et al.
Pubblicazione: (2022)

A More Accurate Approximation of Activation Function with Few Spikes Neurons
di: Jeong, Dayena, et al.
Pubblicazione: (2024)

Mousse: Rectifying the Geometry of Muon with Curvature-Aware Preconditioning
di: Zhang, Yechen, et al.
Pubblicazione: (2026)

Dual Space Preconditioning for Gradient Descent in the Overparameterized Regime
di: Ghane, Reza, et al.
Pubblicazione: (2026)

PROMISE: Preconditioned Stochastic Optimization Methods by Incorporating Scalable Curvature Estimates
di: Frangella, Zachary, et al.
Pubblicazione: (2023)

A Stein Gradient Descent Approach for Doubly Intractable Distributions
di: Lee, Heesang, et al.
Pubblicazione: (2024)

Efficient Curvature-Aware Hypergradient Approximation for Bilevel Optimization
di: Dong, Youran, et al.
Pubblicazione: (2025)

Mirror and Preconditioned Gradient Descent in Wasserstein Space
di: Bonet, Clément, et al.
Pubblicazione: (2024)

Preconditioning for Accelerated Gradient Descent Optimization and Regularization
di: Ye, Qiang
Pubblicazione: (2024)

Efficient Low-Tubal-Rank Tensor Estimation via Alternating Preconditioned Gradient Descent
di: Liu, Zhiyu, et al.
Pubblicazione: (2025)

Papez: Resource-Efficient Speech Separation with Auditory Working Memory
di: Oh, Hyunseok, et al.
Pubblicazione: (2024)

Adaptive Step Sizes for Preconditioned Stochastic Gradient Descent
di: Köhne, Frederik, et al.
Pubblicazione: (2023)

Scalable Mean-Field Variational Inference via Preconditioned Primal-Dual Optimization
di: Lyu, Jinhua, et al.
Pubblicazione: (2026)

Efficient Over-parameterized Matrix Sensing from Noisy Measurements via Alternating Preconditioned Gradient Descent
di: Liu, Zhiyu, et al.
Pubblicazione: (2025)

Preconditioned Gradient Descent for Over-Parameterized Nonconvex Matrix Factorization
di: Zhang, Gavin, et al.
Pubblicazione: (2025)

Diffusion Fine-Tuning via Reparameterized Policy Gradient of the Soft Q-Function
di: Kang, Hyeongyu, et al.
Pubblicazione: (2025)

Mitigating Quantization Errors Due to Activation Spikes in GLU-Based LLMs
di: Yang, Jaewoo, et al.
Pubblicazione: (2024)

Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariance
di: Chen, Yifan, et al.
Pubblicazione: (2023)

DyKAF: Dynamical Kronecker Approximation of the Fisher Information Matrix for Gradient Preconditioning
di: Yudin, Nikolay, et al.
Pubblicazione: (2025)

Inverting the Leverage Score Gradient: An Efficient Approximate Newton Method
di: Li, Chenyang, et al.
Pubblicazione: (2024)

Deep Learning-Enhanced Preconditioning for Efficient Conjugate Gradient Solvers in Large-Scale PDE Systems
di: Li, Rui, et al.
Pubblicazione: (2024)

Geometric Embedding Alignment via Curvature Matching in Transfer Learning
di: Ko, Sung Moon, et al.
Pubblicazione: (2025)

ReMoDetect: Reward Models Recognize Aligned LLM's Generations
di: Lee, Hyunseok, et al.
Pubblicazione: (2024)

Kronecker-factored Approximate Curvature (KFAC) From Scratch
di: Dangel, Felix, et al.
Pubblicazione: (2025)

AGD: an Auto-switchable Optimizer using Stepwise Gradient Difference for Preconditioning Matrix
di: Yue, Yun, et al.
Pubblicazione: (2023)

Ginger: An Efficient Curvature Approximation with Linear Complexity for General Neural Networks
di: Hao, Yongchang, et al.
Pubblicazione: (2024)

Posterior Approximation using Stochastic Gradient Ascent with Adaptive Stepsize
di: Lim, Kart-Leong, et al.
Pubblicazione: (2024)

Efficient Search for Customized Activation Functions with Gradient Descent
di: Strack, Lukas, et al.
Pubblicazione: (2024)

ConditionNET: Learning Preconditions and Effects for Execution Monitoring
di: Sliwowski, Daniel, et al.
Pubblicazione: (2025)

Kronecker-Factored Approximate Curvature for Modern Neural Network Architectures
di: Eschenhagen, Runa, et al.
Pubblicazione: (2023)

A Unifying View of Linear Function Approximation in Off-Policy RL Through Matrix Splitting and Preconditioning
di: Wu, Zechen, et al.
Pubblicazione: (2025)

Universal Sequence Preconditioning
di: Marsden, Annie, et al.
Pubblicazione: (2025)

Preconditioned Gradient Descent for Overparameterized Nonconvex Burer--Monteiro Factorization with Global Optimality Certification
di: Zhang, Gavin, et al.
Pubblicazione: (2022)