Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.04457 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911686025281536 |
|---|---|
| author | Zhao, Guoshenghui Lin, Huawei Zhao, Weijie |
| author_facet | Zhao, Guoshenghui Lin, Huawei Zhao, Weijie |
| contents | Removing specific data influence from large language models (LLMs) remains challenging, as retraining is costly and existing approximate unlearning methods are often unstable. The challenge is exacerbated when the forget set is small or imbalanced. We introduce RapidUn, an influence-driven and parameter-efficient unlearning framework. It first estimates per-sample influence through a fast estimation module, then maps these scores into adaptive update weights that guide selective parameter updates -- forgetting harmful behavior while retaining general knowledge. On Mistral-7B and Llama-3-8B across Dolly-15k and Alpaca-57k, RapidUn achieves up to 100 times higher efficiency than full retraining and consistently outperforms Fisher, GA, and LoReUn on both in-distribution and out-of-distribution forgetting. These results establish influence-guided parameter reweighting as a scalable and interpretable paradigm for LLM unlearning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_04457 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RapidUn: Influence-Driven Parameter Reweighting for Efficient Large Language Model Unlearning Zhao, Guoshenghui Lin, Huawei Zhao, Weijie Computation and Language Removing specific data influence from large language models (LLMs) remains challenging, as retraining is costly and existing approximate unlearning methods are often unstable. The challenge is exacerbated when the forget set is small or imbalanced. We introduce RapidUn, an influence-driven and parameter-efficient unlearning framework. It first estimates per-sample influence through a fast estimation module, then maps these scores into adaptive update weights that guide selective parameter updates -- forgetting harmful behavior while retaining general knowledge. On Mistral-7B and Llama-3-8B across Dolly-15k and Alpaca-57k, RapidUn achieves up to 100 times higher efficiency than full retraining and consistently outperforms Fisher, GA, and LoReUn on both in-distribution and out-of-distribution forgetting. These results establish influence-guided parameter reweighting as a scalable and interpretable paradigm for LLM unlearning. |
| title | RapidUn: Influence-Driven Parameter Reweighting for Efficient Large Language Model Unlearning |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2512.04457 |