Data Attribution for Text-to-Image Models by Unlearning Synthesized Images
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866929721624756224 |
|---|---|
| author | Wang, Sheng-Yu Hertzmann, Aaron Efros, Alexei A. Zhu, Jun-Yan Zhang, Richard |
| author_facet | Wang, Sheng-Yu Hertzmann, Aaron Efros, Alexei A. Zhu, Jun-Yan Zhang, Richard |
| contents | The goal of data attribution for text-to-image models is to identify the training images that most influence the generation of a new image. Influence is defined such that, for a given output, if a model is retrained from scratch without the most influential images, the model would fail to reproduce the same output. Unfortunately, directly searching for these influential images is computationally infeasible, since it would require repeatedly retraining models from scratch. In our work, we propose an efficient data attribution method by simulating unlearning the synthesized image. We achieve this by increasing the training loss on the output image, without catastrophic forgetting of other, unrelated concepts. We then identify training images with significant loss deviations after the unlearning process and label these as influential. We evaluate our method with a computationally intensive but "gold-standard" retraining from scratch and demonstrate our method's advantages over previous methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_09408 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Data Attribution for Text-to-Image Models by Unlearning Synthesized Images Wang, Sheng-Yu Hertzmann, Aaron Efros, Alexei A. Zhu, Jun-Yan Zhang, Richard Computer Vision and Pattern Recognition Machine Learning The goal of data attribution for text-to-image models is to identify the training images that most influence the generation of a new image. Influence is defined such that, for a given output, if a model is retrained from scratch without the most influential images, the model would fail to reproduce the same output. Unfortunately, directly searching for these influential images is computationally infeasible, since it would require repeatedly retraining models from scratch. In our work, we propose an efficient data attribution method by simulating unlearning the synthesized image. We achieve this by increasing the training loss on the output image, without catastrophic forgetting of other, unrelated concepts. We then identify training images with significant loss deviations after the unlearning process and label these as influential. We evaluate our method with a computationally intensive but "gold-standard" retraining from scratch and demonstrate our method's advantages over previous methods. |
| title | Data Attribution for Text-to-Image Models by Unlearning Synthesized Images |
| topic | Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2406.09408 |