An Inversion-based Measure of Memorization for Diffusion Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912512201457664 |
|---|---|
| author | Ma, Zhe Li, Qingming Zhang, Xuhong Du, Tianyu Lin, Ruixiao Wang, Zonghui Ji, Shouling Chen, Wenzhi |
| author_facet | Ma, Zhe Li, Qingming Zhang, Xuhong Du, Tianyu Lin, Ruixiao Wang, Zonghui Ji, Shouling Chen, Wenzhi |
| contents | The past few years have witnessed substantial advances in image generation powered by diffusion models. However, it was shown that diffusion models are susceptible to training data memorization, raising significant concerns regarding copyright infringement and privacy invasion. This study delves into a rigorous analysis of memorization in diffusion models. We introduce InvMM, an inversion-based measure of memorization, which is based on inverting a sensitive latent noise distribution accounting for the replication of an image. For accurate estimation of the measure, we propose an adaptive algorithm that balances the normality and sensitivity of the noise distribution. Comprehensive experiments across four datasets, conducted on both unconditional and text-guided diffusion models, demonstrate that InvMM provides a reliable and complete quantification of memorization. Notably, InvMM is commensurable between samples, reveals the true extent of memorization from an adversarial standpoint and implies how memorization differs from membership. In practice, it serves as an auditing tool for developers to reliably assess the risk of memorization, thereby contributing to the enhancement of trustworthiness and privacy-preserving capabilities of diffusion models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_05846 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | An Inversion-based Measure of Memorization for Diffusion Models Ma, Zhe Li, Qingming Zhang, Xuhong Du, Tianyu Lin, Ruixiao Wang, Zonghui Ji, Shouling Chen, Wenzhi Cryptography and Security Computer Vision and Pattern Recognition The past few years have witnessed substantial advances in image generation powered by diffusion models. However, it was shown that diffusion models are susceptible to training data memorization, raising significant concerns regarding copyright infringement and privacy invasion. This study delves into a rigorous analysis of memorization in diffusion models. We introduce InvMM, an inversion-based measure of memorization, which is based on inverting a sensitive latent noise distribution accounting for the replication of an image. For accurate estimation of the measure, we propose an adaptive algorithm that balances the normality and sensitivity of the noise distribution. Comprehensive experiments across four datasets, conducted on both unconditional and text-guided diffusion models, demonstrate that InvMM provides a reliable and complete quantification of memorization. Notably, InvMM is commensurable between samples, reveals the true extent of memorization from an adversarial standpoint and implies how memorization differs from membership. In practice, it serves as an auditing tool for developers to reliably assess the risk of memorization, thereby contributing to the enhancement of trustworthiness and privacy-preserving capabilities of diffusion models. |
| title | An Inversion-based Measure of Memorization for Diffusion Models |
| topic | Cryptography and Security Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2405.05846 |