Fixing confirmation bias in feature attribution methods via semantic match
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910342643187712 |
|---|---|
| author | Cinà, Giovanni Fernandez-Llaneza, Daniel Deponte, Ludovico Mishra, Nishant Röber, Tabea E. Pezzelle, Sandro Calixto, Iacer Goedhart, Rob Birbil, Ş. İlker |
| author_facet | Cinà, Giovanni Fernandez-Llaneza, Daniel Deponte, Ludovico Mishra, Nishant Röber, Tabea E. Pezzelle, Sandro Calixto, Iacer Goedhart, Rob Birbil, Ş. İlker |
| contents | Feature attribution methods have become a staple method to disentangle the complex behavior of black box models. Despite their success, some scholars have argued that such methods suffer from a serious flaw: they do not allow a reliable interpretation in terms of human concepts. Simply put, visualizing an array of feature contributions is not enough for humans to conclude something about a model's internal representations, and confirmation bias can trick users into false beliefs about model behavior. We argue that a structured approach is required to test whether our hypotheses on the model are confirmed by the feature attributions. This is what we call the "semantic match" between human concepts and (sub-symbolic) explanations. Building on the conceptual framework put forward in Cinà et al. [2023], we propose a structured approach to evaluate semantic match in practice. We showcase the procedure in a suite of experiments spanning tabular and image data, and show how the assessment of semantic match can give insight into both desirable (e.g., focusing on an object relevant for prediction) and undesirable model behaviors (e.g., focusing on a spurious correlation). We couple our experimental results with an analysis on the metrics to measure semantic match, and argue that this approach constitutes the first step towards resolving the issue of confirmation bias in XAI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2307_00897 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Fixing confirmation bias in feature attribution methods via semantic match Cinà, Giovanni Fernandez-Llaneza, Daniel Deponte, Ludovico Mishra, Nishant Röber, Tabea E. Pezzelle, Sandro Calixto, Iacer Goedhart, Rob Birbil, Ş. İlker Machine Learning Artificial Intelligence Feature attribution methods have become a staple method to disentangle the complex behavior of black box models. Despite their success, some scholars have argued that such methods suffer from a serious flaw: they do not allow a reliable interpretation in terms of human concepts. Simply put, visualizing an array of feature contributions is not enough for humans to conclude something about a model's internal representations, and confirmation bias can trick users into false beliefs about model behavior. We argue that a structured approach is required to test whether our hypotheses on the model are confirmed by the feature attributions. This is what we call the "semantic match" between human concepts and (sub-symbolic) explanations. Building on the conceptual framework put forward in Cinà et al. [2023], we propose a structured approach to evaluate semantic match in practice. We showcase the procedure in a suite of experiments spanning tabular and image data, and show how the assessment of semantic match can give insight into both desirable (e.g., focusing on an object relevant for prediction) and undesirable model behaviors (e.g., focusing on a spurious correlation). We couple our experimental results with an analysis on the metrics to measure semantic match, and argue that this approach constitutes the first step towards resolving the issue of confirmation bias in XAI. |
| title | Fixing confirmation bias in feature attribution methods via semantic match |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2307.00897 |