Explainable AI needs formalization
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911574724182016 |
|---|---|
| author | Haufe, Stefan Wilming, Rick Clark, Benedict Zhumagambetov, Rustam Boubekki, Ahcène Martin, Jörg Panknin, Danny |
| author_facet | Haufe, Stefan Wilming, Rick Clark, Benedict Zhumagambetov, Rustam Boubekki, Ahcène Martin, Jörg Panknin, Danny |
| contents | The field of "explainable artificial intelligence" (XAI) seemingly addresses the desire that decisions of machine learning systems should be human-understandable. However, in its current state, XAI itself needs scrutiny. Popular methods cannot reliably answer relevant questions about ML models, their training data, or test inputs, because they systematically attribute importance to input features that are independent of the prediction target. This limits the utility of XAI for diagnosing and correcting data and models, for scientific discovery, and for identifying intervention targets. The fundamental reason for this is that current XAI methods do not address well-defined problems and are not evaluated against targeted criteria of explanation correctness. Researchers should formally define the problems they intend to solve and design methods accordingly. This will lead to diverse use-case-dependent notions of explanation correctness and objective metrics of explanation performance that can be used to validate XAI algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_14590 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Explainable AI needs formalization Haufe, Stefan Wilming, Rick Clark, Benedict Zhumagambetov, Rustam Boubekki, Ahcène Martin, Jörg Panknin, Danny Machine Learning Artificial Intelligence The field of "explainable artificial intelligence" (XAI) seemingly addresses the desire that decisions of machine learning systems should be human-understandable. However, in its current state, XAI itself needs scrutiny. Popular methods cannot reliably answer relevant questions about ML models, their training data, or test inputs, because they systematically attribute importance to input features that are independent of the prediction target. This limits the utility of XAI for diagnosing and correcting data and models, for scientific discovery, and for identifying intervention targets. The fundamental reason for this is that current XAI methods do not address well-defined problems and are not evaluated against targeted criteria of explanation correctness. Researchers should formally define the problems they intend to solve and design methods accordingly. This will lead to diverse use-case-dependent notions of explanation correctness and objective metrics of explanation performance that can be used to validate XAI algorithms. |
| title | Explainable AI needs formalization |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2409.14590 |