Randomized Antipodal Search Done Right for Data Pareto Improvement of LLM Unlearning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913041766940672 |
|---|---|
| author | Liu, Ziwen Lin, Huawei Ran, Yide Zhang, Denghui Xie, Jianwen Li, Chuan Zhao, Weijie Xu, Zhaozhuo |
| author_facet | Liu, Ziwen Lin, Huawei Ran, Yide Zhang, Denghui Xie, Jianwen Li, Chuan Zhao, Weijie Xu, Zhaozhuo |
| contents | Large language models (LLMs) sometimes memorize undesirable knowledge, which must be removed after deployment. Prior work on machine unlearning has focused largely on optimization methods that adjust parameters to enforce forgetting while preserving retention. However, these approaches assume that the forget and retain sets are readily available, which rarely holds in practice. Unlearning is typically triggered by an undesired generation at inference time, making the retrieval of relevant data the central challenge.
We introduce the notion of data Pareto improvement for LLM unlearning, which formalizes how retrieval can expand the achievable trade-off frontier between forgetting and retention. To realize this principle, we propose Randomized Antipodal Search on Linearized Influence Kernel (RASLIK), a retrieval algorithm that combines permutation-projection hashing with randomized antipodal search. RASLIK reduces selection variance, achieves sublinear complexity, and yields a double gain in both quality and efficiency. Across multiple models, datasets, and unlearning algorithms, RASLIK consistently outperforms deterministic baselines and even oracle sampling, establishing randomized search as a principled and scalable solution for data-centric unlearning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_16591 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Randomized Antipodal Search Done Right for Data Pareto Improvement of LLM Unlearning Liu, Ziwen Lin, Huawei Ran, Yide Zhang, Denghui Xie, Jianwen Li, Chuan Zhao, Weijie Xu, Zhaozhuo Machine Learning Artificial Intelligence Large language models (LLMs) sometimes memorize undesirable knowledge, which must be removed after deployment. Prior work on machine unlearning has focused largely on optimization methods that adjust parameters to enforce forgetting while preserving retention. However, these approaches assume that the forget and retain sets are readily available, which rarely holds in practice. Unlearning is typically triggered by an undesired generation at inference time, making the retrieval of relevant data the central challenge. We introduce the notion of data Pareto improvement for LLM unlearning, which formalizes how retrieval can expand the achievable trade-off frontier between forgetting and retention. To realize this principle, we propose Randomized Antipodal Search on Linearized Influence Kernel (RASLIK), a retrieval algorithm that combines permutation-projection hashing with randomized antipodal search. RASLIK reduces selection variance, achieves sublinear complexity, and yields a double gain in both quality and efficiency. Across multiple models, datasets, and unlearning algorithms, RASLIK consistently outperforms deterministic baselines and even oracle sampling, establishing randomized search as a principled and scalable solution for data-centric unlearning. |
| title | Randomized Antipodal Search Done Right for Data Pareto Improvement of LLM Unlearning |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2604.16591 |