Measuring memorization in language models via probabilistic extraction
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908275755188224 |
|---|---|
| author | Hayes, Jamie Swanberg, Marika Chaudhari, Harsh Yona, Itay Shumailov, Ilia Nasr, Milad Choquette-Choo, Christopher A. Lee, Katherine Cooper, A. Feder |
| author_facet | Hayes, Jamie Swanberg, Marika Chaudhari, Harsh Yona, Itay Shumailov, Ilia Nasr, Milad Choquette-Choo, Christopher A. Lee, Katherine Cooper, A. Feder |
| contents | Large language models (LLMs) are susceptible to memorizing training data, raising concerns about the potential extraction of sensitive information at generation time. Discoverable extraction is the most common method for measuring this issue: split a training example into a prefix and suffix, then prompt the LLM with the prefix, and deem the example extractable if the LLM generates the matching suffix using greedy sampling. This definition yields a yes-or-no determination of whether extraction was successful with respect to a single query. Though efficient to compute, we show that this definition is unreliable because it does not account for non-determinism present in more realistic (non-greedy) sampling schemes, for which LLMs produce a range of outputs for the same prompt. We introduce probabilistic discoverable extraction, which, without additional cost, relaxes discoverable extraction by considering multiple queries to quantify the probability of extracting a target sequence. We evaluate our probabilistic measure across different models, sampling schemes, and training-data repetitions, and find that this measure provides more nuanced information about extraction risk compared to traditional discoverable extraction. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_19482 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Measuring memorization in language models via probabilistic extraction Hayes, Jamie Swanberg, Marika Chaudhari, Harsh Yona, Itay Shumailov, Ilia Nasr, Milad Choquette-Choo, Christopher A. Lee, Katherine Cooper, A. Feder Machine Learning Large language models (LLMs) are susceptible to memorizing training data, raising concerns about the potential extraction of sensitive information at generation time. Discoverable extraction is the most common method for measuring this issue: split a training example into a prefix and suffix, then prompt the LLM with the prefix, and deem the example extractable if the LLM generates the matching suffix using greedy sampling. This definition yields a yes-or-no determination of whether extraction was successful with respect to a single query. Though efficient to compute, we show that this definition is unreliable because it does not account for non-determinism present in more realistic (non-greedy) sampling schemes, for which LLMs produce a range of outputs for the same prompt. We introduce probabilistic discoverable extraction, which, without additional cost, relaxes discoverable extraction by considering multiple queries to quantify the probability of extracting a target sequence. We evaluate our probabilistic measure across different models, sampling schemes, and training-data repetitions, and find that this measure provides more nuanced information about extraction risk compared to traditional discoverable extraction. |
| title | Measuring memorization in language models via probabilistic extraction |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2410.19482 |