WaterDrum: Watermarking for Data-centric Unlearning Metric
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908803578986496 |
|---|---|
| author | Lu, Xinyang Niu, Xinyuan Lau, Gregory Kang Ruey Nhung, Bui Thi Cam Sim, Rachael Hwee Ling Himawan, John Russell Wen, Fanyu Foo, Chuan-Sheng Ng, See-Kiong Low, Bryan Kian Hsiang |
| author_facet | Lu, Xinyang Niu, Xinyuan Lau, Gregory Kang Ruey Nhung, Bui Thi Cam Sim, Rachael Hwee Ling Himawan, John Russell Wen, Fanyu Foo, Chuan-Sheng Ng, See-Kiong Low, Bryan Kian Hsiang |
| contents | Large language model (LLM) unlearning is critical in real-world applications where it is necessary to efficiently remove the influence of private, copyrighted, or harmful data from some users. Existing utility-centric unlearning metrics (based on model utility) may fail to accurately evaluate the extent of unlearning in realistic settings such as when the forget and retain sets have semantically similar content and/or retraining the model from scratch on the retain set is impractical. This paper presents the first data-centric unlearning metric for LLMs called WaterDrum that exploits robust text watermarking to overcome these limitations. We introduce new benchmark datasets (with different levels of data similarity) for LLM unlearning that can be used to rigorously evaluate unlearning algorithms via WaterDrum. Our code is available at https://github.com/lululu008/WaterDrum and our new benchmark datasets are released at https://huggingface.co/datasets/Glow-AI/WaterDrum-Ax. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_05064 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | WaterDrum: Watermarking for Data-centric Unlearning Metric Lu, Xinyang Niu, Xinyuan Lau, Gregory Kang Ruey Nhung, Bui Thi Cam Sim, Rachael Hwee Ling Himawan, John Russell Wen, Fanyu Foo, Chuan-Sheng Ng, See-Kiong Low, Bryan Kian Hsiang Machine Learning Large language model (LLM) unlearning is critical in real-world applications where it is necessary to efficiently remove the influence of private, copyrighted, or harmful data from some users. Existing utility-centric unlearning metrics (based on model utility) may fail to accurately evaluate the extent of unlearning in realistic settings such as when the forget and retain sets have semantically similar content and/or retraining the model from scratch on the retain set is impractical. This paper presents the first data-centric unlearning metric for LLMs called WaterDrum that exploits robust text watermarking to overcome these limitations. We introduce new benchmark datasets (with different levels of data similarity) for LLM unlearning that can be used to rigorously evaluate unlearning algorithms via WaterDrum. Our code is available at https://github.com/lululu008/WaterDrum and our new benchmark datasets are released at https://huggingface.co/datasets/Glow-AI/WaterDrum-Ax. |
| title | WaterDrum: Watermarking for Data-centric Unlearning Metric |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2505.05064 |