Benchmarking XAI Explanations with Human-Aligned Evaluations
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908502788669440 |
|---|---|
| author | Kazmierczak, Rémi Azzolin, Steve Berthier, Eloïse Hedström, Anna Delhomme, Patricia Filliat, David Bousquet, Nicolas Frehse, Goran Mancini, Massimiliano Caramiaux, Baptiste Passerini, Andrea Franchi, Gianni |
| author_facet | Kazmierczak, Rémi Azzolin, Steve Berthier, Eloïse Hedström, Anna Delhomme, Patricia Filliat, David Bousquet, Nicolas Frehse, Goran Mancini, Massimiliano Caramiaux, Baptiste Passerini, Andrea Franchi, Gianni |
| contents | We introduce PASTA (Perceptual Assessment System for explanaTion of Artificial Intelligence), a novel human-centric framework for evaluating eXplainable AI (XAI) techniques in computer vision. Our first contribution is the creation of the PASTA-dataset, the first large-scale benchmark that spans a diverse set of models and both saliency-based and concept-based explanation methods. This dataset enables robust, comparative analysis of XAI techniques based on human judgment. Our second contribution is an automated, data-driven benchmark that predicts human preferences using the PASTA-dataset. This scoring called PASTA-score method offers scalable, reliable, and consistent evaluation aligned with human perception. Additionally, our benchmark allows for comparisons between explanations across different modalities, an aspect previously unaddressed. We then propose to apply our scoring method to probe the interpretability of existing models and to build more human interpretable XAI methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_02470 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Benchmarking XAI Explanations with Human-Aligned Evaluations Kazmierczak, Rémi Azzolin, Steve Berthier, Eloïse Hedström, Anna Delhomme, Patricia Filliat, David Bousquet, Nicolas Frehse, Goran Mancini, Massimiliano Caramiaux, Baptiste Passerini, Andrea Franchi, Gianni Computer Vision and Pattern Recognition Artificial Intelligence Human-Computer Interaction We introduce PASTA (Perceptual Assessment System for explanaTion of Artificial Intelligence), a novel human-centric framework for evaluating eXplainable AI (XAI) techniques in computer vision. Our first contribution is the creation of the PASTA-dataset, the first large-scale benchmark that spans a diverse set of models and both saliency-based and concept-based explanation methods. This dataset enables robust, comparative analysis of XAI techniques based on human judgment. Our second contribution is an automated, data-driven benchmark that predicts human preferences using the PASTA-dataset. This scoring called PASTA-score method offers scalable, reliable, and consistent evaluation aligned with human perception. Additionally, our benchmark allows for comparisons between explanations across different modalities, an aspect previously unaddressed. We then propose to apply our scoring method to probe the interpretability of existing models and to build more human interpretable XAI methods. |
| title | Benchmarking XAI Explanations with Human-Aligned Evaluations |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Human-Computer Interaction |
| url | https://arxiv.org/abs/2411.02470 |