Cover Image

Fishers for Free? Approximating the Fisher Information Matrix by Recycling the Squared Gradient Accumulator

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, YuXin, Dangel, Felix, Tam, Derek, Raffel, Colin
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Uncovering Model Processing Strategies with Non-Negative Per-Example Fisher Factorization
by: Matena, Michael, et al.
Published: (2023)

Merging by Matching Models in Task Parameter Subspaces
by: Tam, Derek, et al.
Published: (2023)

An Improved Empirical Fisher Approximation for Natural Gradient Descent
by: Wu, Xiaodong, et al.
Published: (2024)

Kernel Approximation of Fisher-Rao Gradient Flows
by: Zhu, Jia-Jie, et al.
Published: (2024)

DyKAF: Dynamical Kronecker Approximation of the Fisher Information Matrix for Gradient Preconditioning
by: Yudin, Nikolay, et al.
Published: (2025)

Trade-Offs of Diagonal Fisher Information Matrix Estimators
by: Soen, Alexander, et al.
Published: (2024)

Riemannian Laplace Approximation with the Fisher Metric
by: Yu, Hanlin, et al.
Published: (2023)

Model Merging via Data-Free Covariance Estimation
by: Hameed, Marawan Gamal Abdel, et al.
Published: (2026)

FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximation
by: Wu, Zhuguanyu, et al.
Published: (2025)

AttriBoT: A Bag of Tricks for Efficiently Approximating Leave-One-Out Context Attribution
by: Liu, Fengyuan, et al.
Published: (2024)

SOFIM: Stochastic Optimization Using Regularized Fisher Information Matrix
by: Sen, Mrinmay, et al.
Published: (2024)

Rank-1 Approximation of Inverse Fisher for Natural Policy Gradients in Deep Reinforcement Learning
by: Huo, Yingxiao, et al.
Published: (2026)

AdaFisher: Adaptive Second Order Optimization via Fisher Information
by: Gomes, Damien Martins, et al.
Published: (2024)

Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective
by: Lin, Wu, et al.
Published: (2024)

Approximating Simple ReLU Networks based on Spectral Decomposition of Fisher Information
by: Ho, Ka Long Keith, et al.
Published: (2025)

Realistic Evaluation of Model Merging for Compositional Generalization
by: Tam, Derek, et al.
Published: (2024)

Kronecker-Factored Approximate Curvature for Physics-Informed Neural Networks
by: Dangel, Felix, et al.
Published: (2024)

Generalized Fisher-Weighted SVD: Scalable Kronecker-Factored Fisher Approximation for Compressing Large Language Models
by: Chekalina, Viktoriia, et al.
Published: (2025)

FGGM: Fisher-Guided Gradient Masking for Continual Learning
by: Tan, Chao-Hong, et al.
Published: (2026)

Convolutions and More as Einsum: A Tensor Network Perspective with Advances for Second-Order Methods
by: Dangel, Felix
Published: (2023)

Kronecker-factored Approximate Curvature (KFAC) From Scratch
by: Dangel, Felix, et al.
Published: (2025)

Efficient, Multimodal, and Derivative-Free Bayesian Inference With Fisher-Rao Gradient Flows
by: Chen, Yifan, et al.
Published: (2024)

FedFisher: Leveraging Fisher Information for One-Shot Federated Learning
by: Jhunjhunwala, Divyansh, et al.
Published: (2024)

TopoFisher: Learning Topological Summary Statistics by Maximizing Fisher Information
by: Biagetti, Matteo, et al.
Published: (2026)

Dynamic Importance Learning using Fisher Information Matrix (FIM) for Nonlinear Dynamic Mapping
by: Eivaghi, Vahid MohammadZadeh, et al.
Published: (2024)

Lowering PyTorch's Memory Consumption for Selective Differentiation
by: Bhatia, Samarth, et al.
Published: (2024)

FRInGe: Distribution-Space Integrated Gradients with Fisher--Rao Geometry
by: Martino, Gabriele, et al.
Published: (2026)

Improving Energy Natural Gradient Descent through Woodbury, Momentum, and Randomization
by: Guzmán-Cordero, Andrés, et al.
Published: (2025)

Approximately optimal domain adaptation with Fisher's Linear Discriminant
by: Helm, Hayden S., et al.
Published: (2023)

Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFAC
by: Lin, Wu, et al.
Published: (2023)

Fisher-Orthogonal Projected Natural Gradient Descent for Continual Learning
by: Garg, Ishir, et al.
Published: (2026)

Approximations to the Fisher Information Metric of Deep Generative Models for Out-Of-Distribution Detection
by: Dauncey, Sam, et al.
Published: (2024)

Efficient Bilevel Optimization with KFAC-Based Hypergradients
by: Liao, Disen, et al.
Published: (2026)

Data-Free Layer-Adaptive Merging via Fisher Information for Long-to-Short Reasoning LLMs
by: Xia, Tian
Published: (2026)

Approximating Families of Sharp Solutions to Fisher's Equation with Physics-Informed Neural Networks
by: Rohrhofer, Franz M., et al.
Published: (2024)

Efficiently Estimating Data Efficiency for Language Model Fine-tuning
by: Je, Gyung Hyun, et al.
Published: (2025)

FIC-TSC: Learning Time Series Classification with Fisher Information Constraint
by: Chen, Xiwen, et al.
Published: (2025)

Multi-Objective Optimization via Wasserstein-Fisher-Rao Gradient Flow
by: Ren, Yinuo, et al.
Published: (2023)

Revisiting Scalable Hessian Diagonal Approximations for Applications in Reinforcement Learning
by: Elsayed, Mohamed, et al.
Published: (2024)

Matrix-Free Least Squares Solvers: Values, Gradients, and What to Do With Them
by: Roy, Hrittik, et al.
Published: (2025)