Shape of Memory: a Geometric Analysis of Machine Unlearning in Second-Order Optimizers
Fuente:
arXiv
Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908991723929600 |
|---|---|
| author | Stewart, Kennon |
| author_facet | Stewart, Kennon |
| contents | We argue that current definitions of machine unlearning are underspecified for second-order optimizers. We compare first-order and second-order learners for their ability to handle the data deletion task with varying degrees of eigendecomposition to mimic the loss model memory. While both first and second-order methods realign with the ideal counterfactul in terms of performance and gradient, the second-order optimizer shows significant volatility in the optimizer state. This indicates residual information, supposedly deleted, that isn't detectable by first-order analysis. Various eigendecay treatments show that stability and information loss is regained only under controlled state pertubation where geometric information (or memory) is erased. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_23046 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Shape of Memory: a Geometric Analysis of Machine Unlearning in Second-Order Optimizers Stewart, Kennon Machine Learning Information Theory Social and Information Networks We argue that current definitions of machine unlearning are underspecified for second-order optimizers. We compare first-order and second-order learners for their ability to handle the data deletion task with varying degrees of eigendecomposition to mimic the loss model memory. While both first and second-order methods realign with the ideal counterfactul in terms of performance and gradient, the second-order optimizer shows significant volatility in the optimizer state. This indicates residual information, supposedly deleted, that isn't detectable by first-order analysis. Various eigendecay treatments show that stability and information loss is regained only under controlled state pertubation where geometric information (or memory) is erased. |
| title | Shape of Memory: a Geometric Analysis of Machine Unlearning in Second-Order Optimizers |
| topic | Machine Learning Information Theory Social and Information Networks |
| url | https://arxiv.org/abs/2604.23046 |