DEPICT: Diffusion-Enabled Permutation Importance for Image Classification Tasks
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911962222297088 |
|---|---|
| author | Jabbour, Sarah Kondas, Gregory Kazerooni, Ella Sjoding, Michael Fouhey, David Wiens, Jenna |
| author_facet | Jabbour, Sarah Kondas, Gregory Kazerooni, Ella Sjoding, Michael Fouhey, David Wiens, Jenna |
| contents | We propose a permutation-based explanation method for image classifiers. Current image-model explanations like activation maps are limited to instance-based explanations in the pixel space, making it difficult to understand global model behavior. In contrast, permutation based explanations for tabular data classifiers measure feature importance by comparing model performance on data before and after permuting a feature. We propose an explanation method for image-based models that permutes interpretable concepts across dataset images. Given a dataset of images labeled with specific concepts like captions, we permute a concept across examples in the text space and then generate images via a text-conditioned diffusion model. Feature importance is then reflected by the change in model performance relative to unpermuted data. When applied to a set of concepts, the method generates a ranking of feature importance. We show this approach recovers underlying model feature importance on synthetic and real-world image classification tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_14509 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | DEPICT: Diffusion-Enabled Permutation Importance for Image Classification Tasks Jabbour, Sarah Kondas, Gregory Kazerooni, Ella Sjoding, Michael Fouhey, David Wiens, Jenna Computer Vision and Pattern Recognition Artificial Intelligence We propose a permutation-based explanation method for image classifiers. Current image-model explanations like activation maps are limited to instance-based explanations in the pixel space, making it difficult to understand global model behavior. In contrast, permutation based explanations for tabular data classifiers measure feature importance by comparing model performance on data before and after permuting a feature. We propose an explanation method for image-based models that permutes interpretable concepts across dataset images. Given a dataset of images labeled with specific concepts like captions, we permute a concept across examples in the text space and then generate images via a text-conditioned diffusion model. Feature importance is then reflected by the change in model performance relative to unpermuted data. When applied to a set of concepts, the method generates a ranking of feature importance. We show this approach recovers underlying model feature importance on synthetic and real-world image classification tasks. |
| title | DEPICT: Diffusion-Enabled Permutation Importance for Image Classification Tasks |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2407.14509 |