Exploring Memorization in Fine-tuned Language Models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911782455476224 |
|---|---|
| author | Zeng, Shenglai Li, Yaxin Ren, Jie Liu, Yiding Xu, Han He, Pengfei Xing, Yue Wang, Shuaiqiang Tang, Jiliang Yin, Dawei |
| author_facet | Zeng, Shenglai Li, Yaxin Ren, Jie Liu, Yiding Xu, Han He, Pengfei Xing, Yue Wang, Shuaiqiang Tang, Jiliang Yin, Dawei |
| contents | Large language models (LLMs) have shown great capabilities in various tasks but also exhibited memorization of training data, raising tremendous privacy and copyright concerns. While prior works have studied memorization during pre-training, the exploration of memorization during fine-tuning is rather limited. Compared to pre-training, fine-tuning typically involves more sensitive data and diverse objectives, thus may bring distinct privacy risks and unique memorization behaviors. In this work, we conduct the first comprehensive analysis to explore language models' (LMs) memorization during fine-tuning across tasks. Our studies with open-sourced and our own fine-tuned LMs across various tasks indicate that memorization presents a strong disparity among different fine-tuning tasks. We provide an intuitive explanation of this task disparity via sparse coding theory and unveil a strong correlation between memorization and attention score distribution. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_06714 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Exploring Memorization in Fine-tuned Language Models Zeng, Shenglai Li, Yaxin Ren, Jie Liu, Yiding Xu, Han He, Pengfei Xing, Yue Wang, Shuaiqiang Tang, Jiliang Yin, Dawei Artificial Intelligence Computation and Language Machine Learning Large language models (LLMs) have shown great capabilities in various tasks but also exhibited memorization of training data, raising tremendous privacy and copyright concerns. While prior works have studied memorization during pre-training, the exploration of memorization during fine-tuning is rather limited. Compared to pre-training, fine-tuning typically involves more sensitive data and diverse objectives, thus may bring distinct privacy risks and unique memorization behaviors. In this work, we conduct the first comprehensive analysis to explore language models' (LMs) memorization during fine-tuning across tasks. Our studies with open-sourced and our own fine-tuned LMs across various tasks indicate that memorization presents a strong disparity among different fine-tuning tasks. We provide an intuitive explanation of this task disparity via sparse coding theory and unveil a strong correlation between memorization and attention score distribution. |
| title | Exploring Memorization in Fine-tuned Language Models |
| topic | Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2310.06714 |