Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.26544 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917300166197248 |
|---|---|
| author | Kreer, Philipp Alexander Wu, Wilson Adam, Maxwell Furman, Zach Hoogland, Jesse |
| author_facet | Kreer, Philipp Alexander Wu, Wilson Adam, Maxwell Furman, Zach Hoogland, Jesse |
| contents | Classical influence functions face significant challenges when applied to deep neural networks, primarily due to non-invertible Hessians and high-dimensional parameter spaces. We propose the local Bayesian influence function (BIF), an extension of classical influence functions that replaces Hessian inversion with loss landscape statistics that can be estimated via stochastic-gradient MCMC sampling. This Hessian-free approach captures higher-order interactions among parameters and scales efficiently to neural networks with billions of parameters. We demonstrate state-of-the-art results on predicting retraining experiments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_26544 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Bayesian Influence Functions for Hessian-Free Data Attribution Kreer, Philipp Alexander Wu, Wilson Adam, Maxwell Furman, Zach Hoogland, Jesse Machine Learning Classical influence functions face significant challenges when applied to deep neural networks, primarily due to non-invertible Hessians and high-dimensional parameter spaces. We propose the local Bayesian influence function (BIF), an extension of classical influence functions that replaces Hessian inversion with loss landscape statistics that can be estimated via stochastic-gradient MCMC sampling. This Hessian-free approach captures higher-order interactions among parameters and scales efficiently to neural networks with billions of parameters. We demonstrate state-of-the-art results on predicting retraining experiments. |
| title | Bayesian Influence Functions for Hessian-Free Data Attribution |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2509.26544 |