Improving Interpretability and Robustness for the Detection of AI-Generated Images
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914843856994304 |
|---|---|
| author | Gaintseva, Tatiana Kushnareva, Laida Magai, German Piontkovskaya, Irina Nikolenko, Sergey Benning, Martin Barannikov, Serguei Slabaugh, Gregory |
| author_facet | Gaintseva, Tatiana Kushnareva, Laida Magai, German Piontkovskaya, Irina Nikolenko, Sergey Benning, Martin Barannikov, Serguei Slabaugh, Gregory |
| contents | With growing abilities of generative models, artificial content detection becomes an increasingly important and difficult task. However, all popular approaches to this problem suffer from poor generalization across domains and generative models. In this work, we focus on the robustness of AI-generated image (AIGI) detectors. We analyze existing state-of-the-art AIGI detection methods based on frozen CLIP embeddings and show how to interpret them, shedding light on how images produced by various AI generators differ from real ones. Next we propose two ways to improve robustness: based on removing harmful components of the embedding vector and based on selecting the best performing attention heads in the image encoder model. Our methods increase the mean out-of-distribution (OOD) classification score by up to 6% for cross-model transfer. We also propose a new dataset for AIGI detection and use it in our evaluation; we believe this dataset will help boost further research. The dataset and code are provided as a supplement. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_15035 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Improving Interpretability and Robustness for the Detection of AI-Generated Images Gaintseva, Tatiana Kushnareva, Laida Magai, German Piontkovskaya, Irina Nikolenko, Sergey Benning, Martin Barannikov, Serguei Slabaugh, Gregory Computer Vision and Pattern Recognition With growing abilities of generative models, artificial content detection becomes an increasingly important and difficult task. However, all popular approaches to this problem suffer from poor generalization across domains and generative models. In this work, we focus on the robustness of AI-generated image (AIGI) detectors. We analyze existing state-of-the-art AIGI detection methods based on frozen CLIP embeddings and show how to interpret them, shedding light on how images produced by various AI generators differ from real ones. Next we propose two ways to improve robustness: based on removing harmful components of the embedding vector and based on selecting the best performing attention heads in the image encoder model. Our methods increase the mean out-of-distribution (OOD) classification score by up to 6% for cross-model transfer. We also propose a new dataset for AIGI detection and use it in our evaluation; we believe this dataset will help boost further research. The dataset and code are provided as a supplement. |
| title | Improving Interpretability and Robustness for the Detection of AI-Generated Images |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2406.15035 |