Cover Image

Kronecker-factored Approximate Curvature (KFAC) From Scratch

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Dangel, Felix, Mucsányi, Bálint, Weber, Tobias, Eschenhagen, Runa
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFAC
by: Lin, Wu, et al.
Published: (2023)

Kronecker-Factored Approximate Curvature for Modern Neural Network Architectures
by: Eschenhagen, Runa, et al.
Published: (2023)

Efficient Bilevel Optimization with KFAC-Based Hypergradients
by: Liao, Disen, et al.
Published: (2026)

Spectral-factorized Positive-definite Curvature Learning for NN Training
by: Lin, Wu, et al.
Published: (2025)

Kronecker-Factored Approximate Curvature for Physics-Informed Neural Networks
by: Dangel, Felix, et al.
Published: (2024)

Position: Curvature Matrices Should Be Democratized via Linear Operators
by: Dangel, Felix, et al.
Published: (2025)

Better Hessians Matter: Studying the Impact of Curvature Approximations in Influence Functions
by: Hong, Steve, et al.
Published: (2025)

Rethinking Approximate Gaussian Inference in Classification
by: Mucsányi, Bálint, et al.
Published: (2025)

Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective
by: Lin, Wu, et al.
Published: (2024)

Understanding and Improving Shampoo and SOAP via Kullback-Leibler Minimization
by: Lin, Wu, et al.
Published: (2025)

laplax -- Laplace Approximations with JAX
by: Weber, Tobias, et al.
Published: (2025)

Geometric Gaussian Approximations of Probability Distributions
by: Da Costa, Nathaël, et al.
Published: (2025)

Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Tasks
by: Mucsányi, Bálint, et al.
Published: (2024)

Logit Reweighting for Topic-Focused Summarization
by: Braun, Joschka, et al.
Published: (2025)

Debiasing Mini-Batch Quadratics for Applications in Deep Learning
by: Tatzel, Lukas, et al.
Published: (2024)

Convolutions and More as Einsum: A Tensor Network Perspective with Advances for Second-Order Methods
by: Dangel, Felix
Published: (2023)

Dynamics Reveals Structure: Challenging the Linear Propagation Assumption
by: Chang, Hoyeon, et al.
Published: (2026)

Lowering PyTorch's Memory Consumption for Selective Differentiation
by: Bhatia, Samarth, et al.
Published: (2024)

Fishers for Free? Approximating the Fisher Information Matrix by Recycling the Squared Gradient Accumulator
by: Li, YuXin, et al.
Published: (2025)

A New Way: Kronecker-Factored Approximate Curvature Deep Hedging and its Benefits
by: Enkhbayar, Tsogt-Ochir
Published: (2024)

Revisiting Scalable Hessian Diagonal Approximations for Applications in Reinforcement Learning
by: Elsayed, Mohamed, et al.
Published: (2024)

Clarifying Shampoo: Adapting Spectral Descent to Stochasticity and the Parameter Trajectory
by: Eschenhagen, Runa, et al.
Published: (2026)

On the Disconnect Between Theory and Practice of Neural Networks: Limits of the NTK Perspective
by: Wenger, Jonathan, et al.
Published: (2023)

Dataless Weight Disentanglement in Task Arithmetic via Kronecker-Factored Approximate Curvature
by: Porrello, Angelo, et al.
Published: (2026)

Skill Learning via Policy Diversity Yields Identifiable Representations for Reinforcement Learning
by: Reizinger, Patrik, et al.
Published: (2025)

Influence Functions for Scalable Data Attribution in Diffusion Models
by: Mlodozeniec, Bruno, et al.
Published: (2024)

What Does It Mean to Be a Transformer? Insights from a Theoretical Hessian Analysis
by: Ormaniec, Weronika, et al.
Published: (2024)

Hide & Seek: Transformer Symmetries Obscure Sharpness & Riemannian Geometry Finds It
by: da Silva, Marvin F., et al.
Published: (2025)

Collapsing Taylor Mode Automatic Differentiation
by: Dangel, Felix, et al.
Published: (2025)

Purifying Shampoo: Investigating Shampoo's Heuristics by Decomposing its Preconditioner
by: Eschenhagen, Runa, et al.
Published: (2025)

Analysis and Approximate Inference of Large Random Kronecker Graphs
by: Liao, Zhenyu, et al.
Published: (2023)

Improving Energy Natural Gradient Descent through Woodbury, Momentum, and Randomization
by: Guzmán-Cordero, Andrés, et al.
Published: (2025)

Weaver: Kronecker Product Approximations of Spatiotemporal Attention for Traffic Network Forecasting
by: Cheong, Christopher, et al.
Published: (2025)

Sketching Low-Rank Plus Diagonal Matrices
by: Fernandez, Andres, et al.
Published: (2025)

Generalizing the Geometry of Model Merging Through Frechet Averages
by: da Silva, Marvin F., et al.
Published: (2026)

Regression for matrix-valued data via Kronecker products factorization
by: Chen, Yin-Jen, et al.
Published: (2024)

Reparametrizing Shampoo and SOAP for Subspace Basis Updates and BFloat16 Storage
by: Milligan, Alan, et al.
Published: (2026)

Effective Structural Encodings via Local Curvature Profiles
by: Fesser, Lukas, et al.
Published: (2023)

Newton Losses: Using Curvature Information for Learning with Differentiable Algorithms
by: Petersen, Felix, et al.
Published: (2024)

Quantum Speedup for Spectral Approximation of Kronecker Products
by: Gao, Yeqi, et al.
Published: (2024)