Towards Evaluation for Real-World LLM Unlearning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916877709606912 |
|---|---|
| author | Miao, Ke Hu, Yuke Li, Xiaochen Bao, Wenjie Liu, Zhihao Qin, Zhan Ren, Kui |
| author_facet | Miao, Ke Hu, Yuke Li, Xiaochen Bao, Wenjie Liu, Zhihao Qin, Zhan Ren, Kui |
| contents | This paper analyzes the limitations of existing unlearning evaluation metrics in terms of practicality, exactness, and robustness in real-world LLM unlearning scenarios. To overcome these limitations, we propose a new metric called Distribution Correction-based Unlearning Evaluation (DCUE). It identifies core tokens and corrects distributional biases in their confidence scores using a validation set. The evaluation results are quantified using the Kolmogorov-Smirnov test. Experimental results demonstrate that DCUE overcomes the limitations of existing metrics, which also guides the design of more practical and reliable unlearning algorithms in the future. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_01324 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Evaluation for Real-World LLM Unlearning Miao, Ke Hu, Yuke Li, Xiaochen Bao, Wenjie Liu, Zhihao Qin, Zhan Ren, Kui Artificial Intelligence This paper analyzes the limitations of existing unlearning evaluation metrics in terms of practicality, exactness, and robustness in real-world LLM unlearning scenarios. To overcome these limitations, we propose a new metric called Distribution Correction-based Unlearning Evaluation (DCUE). It identifies core tokens and corrects distributional biases in their confidence scores using a validation set. The evaluation results are quantified using the Kolmogorov-Smirnov test. Experimental results demonstrate that DCUE overcomes the limitations of existing metrics, which also guides the design of more practical and reliable unlearning algorithms in the future. |
| title | Towards Evaluation for Real-World LLM Unlearning |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2508.01324 |