Understanding Memorisation in LLMs: Dynamics, Influencing Factors, and Implications
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910544890429440 |
|---|---|
| author | Speicher, Till Khan, Mohammad Aflah Wu, Qinyuan Nanda, Vedant Das, Soumi Ghosh, Bishwamittra Gummadi, Krishna P. Terzi, Evimaria |
| author_facet | Speicher, Till Khan, Mohammad Aflah Wu, Qinyuan Nanda, Vedant Das, Soumi Ghosh, Bishwamittra Gummadi, Krishna P. Terzi, Evimaria |
| contents | Understanding whether and to what extent large language models (LLMs) have memorised training data has important implications for the reliability of their output and the privacy of their training data. In order to cleanly measure and disentangle memorisation from other phenomena (e.g. in-context learning), we create an experimental framework that is based on repeatedly exposing LLMs to random strings. Our framework allows us to better understand the dynamics, i.e., the behaviour of the model, when repeatedly exposing it to random strings. Using our framework, we make several striking observations: (a) we find consistent phases of the dynamics across families of models (Pythia, Phi and Llama2), (b) we identify factors that make some strings easier to memorise than others, and (c) we identify the role of local prefixes and global context in memorisation. We also show that sequential exposition to different random strings has a significant effect on memorisation. Our results, often surprising, have significant downstream implications in the study and usage of LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_19262 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Understanding Memorisation in LLMs: Dynamics, Influencing Factors, and Implications Speicher, Till Khan, Mohammad Aflah Wu, Qinyuan Nanda, Vedant Das, Soumi Ghosh, Bishwamittra Gummadi, Krishna P. Terzi, Evimaria Computation and Language Machine Learning Understanding whether and to what extent large language models (LLMs) have memorised training data has important implications for the reliability of their output and the privacy of their training data. In order to cleanly measure and disentangle memorisation from other phenomena (e.g. in-context learning), we create an experimental framework that is based on repeatedly exposing LLMs to random strings. Our framework allows us to better understand the dynamics, i.e., the behaviour of the model, when repeatedly exposing it to random strings. Using our framework, we make several striking observations: (a) we find consistent phases of the dynamics across families of models (Pythia, Phi and Llama2), (b) we identify factors that make some strings easier to memorise than others, and (c) we identify the role of local prefixes and global context in memorisation. We also show that sequential exposition to different random strings has a significant effect on memorisation. Our results, often surprising, have significant downstream implications in the study and usage of LLMs. |
| title | Understanding Memorisation in LLMs: Dynamics, Influencing Factors, and Implications |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2407.19262 |