Understanding and Mitigating Memorization in Generative Models via Sharpness of Probability Landscapes
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915450903855104 |
|---|---|
| author | Jeon, Dongjae Kim, Dueun No, Albert |
| author_facet | Jeon, Dongjae Kim, Dueun No, Albert |
| contents | In this paper, we introduce a geometric framework to analyze memorization in diffusion models through the sharpness of the log probability density. We mathematically justify a previously proposed score-difference-based memorization metric by demonstrating its effectiveness in quantifying sharpness. Additionally, we propose a novel memorization metric that captures sharpness at the initial stage of image generation in latent diffusion models, offering early insights into potential memorization. Leveraging this metric, we develop a mitigation strategy that optimizes the initial noise of the generation process using a sharpness-aware regularization term. The code is publicly available at https://github.com/Dongjae0324/sharpness_memorization_diffusion. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_04140 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Understanding and Mitigating Memorization in Generative Models via Sharpness of Probability Landscapes Jeon, Dongjae Kim, Dueun No, Albert Machine Learning Artificial Intelligence In this paper, we introduce a geometric framework to analyze memorization in diffusion models through the sharpness of the log probability density. We mathematically justify a previously proposed score-difference-based memorization metric by demonstrating its effectiveness in quantifying sharpness. Additionally, we propose a novel memorization metric that captures sharpness at the initial stage of image generation in latent diffusion models, offering early insights into potential memorization. Leveraging this metric, we develop a mitigation strategy that optimizes the initial noise of the generation process using a sharpness-aware regularization term. The code is publicly available at https://github.com/Dongjae0324/sharpness_memorization_diffusion. |
| title | Understanding and Mitigating Memorization in Generative Models via Sharpness of Probability Landscapes |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2412.04140 |