ZJUKLAB at SemEval-2025 Task 4: Unlearning via Model Merging
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910914924511232 |
|---|---|
| author | Xu, Haoming Wang, Shuxun Zhao, Yanqiu Zhong, Yi Jiang, Ziyan Zhao, Ningyuan Deng, Shumin Chen, Huajun Zhang, Ningyu |
| author_facet | Xu, Haoming Wang, Shuxun Zhao, Yanqiu Zhong, Yi Jiang, Ziyan Zhao, Ningyuan Deng, Shumin Chen, Huajun Zhang, Ningyu |
| contents | This paper presents the ZJUKLAB team's submission for SemEval-2025 Task 4: Unlearning Sensitive Content from Large Language Models. This task aims to selectively erase sensitive knowledge from large language models, avoiding both over-forgetting and under-forgetting issues. We propose an unlearning system that leverages Model Merging (specifically TIES-Merging), combining two specialized models into a more balanced unlearned model. Our system achieves competitive results, ranking second among 26 teams, with an online score of 0.944 for Task Aggregate and 0.487 for overall Aggregate. In this paper, we also conduct local experiments and perform a comprehensive analysis of the unlearning process, examining performance trajectories, loss dynamics, and weight perspectives, along with several supplementary experiments, to understand the effectiveness of our method. Furthermore, we analyze the shortcomings of our method and evaluation metrics, emphasizing that MIA scores and ROUGE-based metrics alone are insufficient to fully evaluate successful unlearning. Finally, we emphasize the need for more comprehensive evaluation methodologies and rethinking of unlearning objectives in future research. Code is available at https://github.com/zjunlp/unlearn/tree/main/semeval25. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_21088 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ZJUKLAB at SemEval-2025 Task 4: Unlearning via Model Merging Xu, Haoming Wang, Shuxun Zhao, Yanqiu Zhong, Yi Jiang, Ziyan Zhao, Ningyuan Deng, Shumin Chen, Huajun Zhang, Ningyu Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning Multimedia This paper presents the ZJUKLAB team's submission for SemEval-2025 Task 4: Unlearning Sensitive Content from Large Language Models. This task aims to selectively erase sensitive knowledge from large language models, avoiding both over-forgetting and under-forgetting issues. We propose an unlearning system that leverages Model Merging (specifically TIES-Merging), combining two specialized models into a more balanced unlearned model. Our system achieves competitive results, ranking second among 26 teams, with an online score of 0.944 for Task Aggregate and 0.487 for overall Aggregate. In this paper, we also conduct local experiments and perform a comprehensive analysis of the unlearning process, examining performance trajectories, loss dynamics, and weight perspectives, along with several supplementary experiments, to understand the effectiveness of our method. Furthermore, we analyze the shortcomings of our method and evaluation metrics, emphasizing that MIA scores and ROUGE-based metrics alone are insufficient to fully evaluate successful unlearning. Finally, we emphasize the need for more comprehensive evaluation methodologies and rethinking of unlearning objectives in future research. Code is available at https://github.com/zjunlp/unlearn/tree/main/semeval25. |
| title | ZJUKLAB at SemEval-2025 Task 4: Unlearning via Model Merging |
| topic | Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning Multimedia |
| url | https://arxiv.org/abs/2503.21088 |