Cover Image

Better Hessians Matter: Studying the Impact of Curvature Approximations in Influence Functions

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Hong, Steve, Eschenhagen, Runa, Mlodozeniec, Bruno, Turner, Richard
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Influence Functions for Scalable Data Attribution in Diffusion Models
by: Mlodozeniec, Bruno, et al.
Published: (2024)

Kronecker-Factored Approximate Curvature for Modern Neural Network Architectures
by: Eschenhagen, Runa, et al.
Published: (2023)

Kronecker-factored Approximate Curvature (KFAC) From Scratch
by: Dangel, Felix, et al.
Published: (2025)

Spectral-factorized Positive-definite Curvature Learning for NN Training
by: Lin, Wu, et al.
Published: (2025)

Probabilistic Modelling is Sufficient for Causal Inference
by: Mlodozeniec, Bruno, et al.
Published: (2025)

Position: Curvature Matrices Should Be Democratized via Linear Operators
by: Dangel, Felix, et al.
Published: (2025)

Distributional Training Data Attribution: What do Influence Functions Sample?
by: Mlodozeniec, Bruno, et al.
Published: (2025)

Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective
by: Lin, Wu, et al.
Published: (2024)

Incremental Transformer Neural Processes
by: Mortimer, Philip, et al.
Published: (2026)

Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFAC
by: Lin, Wu, et al.
Published: (2023)

Purifying Shampoo: Investigating Shampoo's Heuristics by Decomposing its Preconditioner
by: Eschenhagen, Runa, et al.
Published: (2025)

Clarifying Shampoo: Adapting Spectral Descent to Stochasticity and the Parameter Trajectory
by: Eschenhagen, Runa, et al.
Published: (2026)

Denoising Diffusion Probabilistic Models in Six Simple Steps
by: Turner, Richard E., et al.
Published: (2024)

Bayesian Influence Functions for Hessian-Free Data Attribution
by: Kreer, Philipp Alexander, et al.
Published: (2025)

Revisit, Extend, and Enhance Hessian-Free Influence Functions
by: Yang, Ziao, et al.
Published: (2024)

Understanding and Improving Shampoo and SOAP via Kullback-Leibler Minimization
by: Lin, Wu, et al.
Published: (2025)

Better Training Data Attribution via Better Inverse Hessian-Vector Products
by: Wang, Andrew, et al.
Published: (2025)

Revisiting Zeroth-Order Hessian Approximation: A Single-Step Policy Optimization Lens
by: Qiu, Junbin, et al.
Published: (2026)

A Generative Model of Symmetry Transformations
by: Allingham, James Urquhart, et al.
Published: (2024)

Warm Start Marginal Likelihood Optimisation for Iterative Gaussian Processes
by: Lin, Jihao Andreas, et al.
Published: (2024)

Implicit meta-learning may lead language models to trust more reliable sources
by: Krasheninnikov, Dmitrii, et al.
Published: (2023)

Gradient-Normalized Smoothness for Optimization with Approximate Hessians
by: Semenov, Andrei, et al.
Published: (2025)

Rotary Position Encodings for Graphs
by: Reid, Isaac, et al.
Published: (2025)

Improving Linear System Solvers for Hyperparameter Optimisation in Iterative Gaussian Processes
by: Lin, Jihao Andreas, et al.
Published: (2024)

FAGH: Accelerating Federated Learning with Approximated Global Hessian
by: Sen, Mrinmay, et al.
Published: (2024)

Subspace-based Approximate Hessian Method for Zeroth-Order Optimization
by: Kim, Dongyoon, et al.
Published: (2025)

Revisiting Scalable Hessian Diagonal Approximations for Applications in Reinforcement Learning
by: Elsayed, Mohamed, et al.
Published: (2024)

Approximately Equivariant Neural Processes
by: Ashman, Matthew, et al.
Published: (2024)

Projected Hessian Learning: Fast Curvature Supervision for Accurate Machine-Learning Interatomic Potentials
by: Rodriguez, Austin, et al.
Published: (2026)

Mimicking Better by Matching the Approximate Action Distribution
by: Ramos, João A. Cândido, et al.
Published: (2023)

Completed Hyperparameter Transfer across Modules, Width, Depth, Batch and Duration
by: Mlodozeniec, Bruno, et al.
Published: (2025)

SASSHA: Sharpness-aware Adaptive Second-order Optimization with Stable Hessian Approximation
by: Shin, Dahun, et al.
Published: (2025)

Unraveling the Hessian: A Key to Smooth Convergence in Loss Function Landscapes
by: Kiselev, Nikita, et al.
Published: (2024)

The Approximate Fisher Influence Function: Faster Estimation of Data Influence in Statistical Models
by: Lev, Omri, et al.
Published: (2024)

Closing the Curvature Gap: Full Transformer Hessians and Their Implications for Scaling Laws
by: Petrov, Egor, et al.
Published: (2025)

MAC: An Efficient Gradient Preconditioning using Mean Activation Approximated Curvature
by: Seung, Hyunseok, et al.
Published: (2025)

Efficient Curvature-Aware Hypergradient Approximation for Bilevel Optimization
by: Dong, Youran, et al.
Published: (2025)

Unbiased Online Curvature Approximation for Regularized Graph Continual Learning
by: Yin, Jie, et al.
Published: (2025)

Kronecker-Factored Approximate Curvature for Physics-Informed Neural Networks
by: Dangel, Felix, et al.
Published: (2024)

Online estimation of the inverse of the Hessian for stochastic optimization with application to universal stochastic Newton algorithms
by: Godichon-Baggioni, Antoine, et al.
Published: (2024)