Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.24832 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913899741184000 |
|---|---|
| author | Morris, John X. Sitawarin, Chawin Guo, Chuan Kokhlikyan, Narine Suh, G. Edward Rush, Alexander M. Chaudhuri, Kamalika Mahloujifar, Saeed |
| author_facet | Morris, John X. Sitawarin, Chawin Guo, Chuan Kokhlikyan, Narine Suh, G. Edward Rush, Alexander M. Chaudhuri, Kamalika Mahloujifar, Saeed |
| contents | We propose a new method for estimating how much a model knows about a datapoint and use it to measure the capacity of modern language models. Prior studies of language model memorization have struggled to disentangle memorization from generalization. We formally separate memorization into two components: unintended memorization, the information a model contains about a specific dataset, and generalization, the information a model contains about the true data-generation process. When we completely eliminate generalization, we can compute the total memorization, which provides an estimate of model capacity: our measurements estimate that GPT-style models have a capacity of approximately 3.6 bits per parameter. We train language models on datasets of increasing size and observe that models memorize until their capacity fills, at which point "grokking" begins, and unintended memorization decreases as models begin to generalize. We train hundreds of transformer language models ranging from $500K$ to $1.5B$ parameters and produce a series of scaling laws relating model capacity and data size to membership inference. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_24832 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | How much do language models memorize? Morris, John X. Sitawarin, Chawin Guo, Chuan Kokhlikyan, Narine Suh, G. Edward Rush, Alexander M. Chaudhuri, Kamalika Mahloujifar, Saeed Computation and Language We propose a new method for estimating how much a model knows about a datapoint and use it to measure the capacity of modern language models. Prior studies of language model memorization have struggled to disentangle memorization from generalization. We formally separate memorization into two components: unintended memorization, the information a model contains about a specific dataset, and generalization, the information a model contains about the true data-generation process. When we completely eliminate generalization, we can compute the total memorization, which provides an estimate of model capacity: our measurements estimate that GPT-style models have a capacity of approximately 3.6 bits per parameter. We train language models on datasets of increasing size and observe that models memorize until their capacity fills, at which point "grokking" begins, and unintended memorization decreases as models begin to generalize. We train hundreds of transformer language models ranging from $500K$ to $1.5B$ parameters and produce a series of scaling laws relating model capacity and data size to membership inference. |
| title | How much do language models memorize? |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2505.24832 |