Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2310.06514 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911776773242880 |
|---|---|
| author | Zhang, Yang Li, Yawei Brown, Hannah Rezaei, Mina Bischl, Bernd Torr, Philip Khakzar, Ashkan Kawaguchi, Kenji |
| author_facet | Zhang, Yang Li, Yawei Brown, Hannah Rezaei, Mina Bischl, Bernd Torr, Philip Khakzar, Ashkan Kawaguchi, Kenji |
| contents | Feature attribution explains neural network outputs by identifying relevant input features. The attribution has to be faithful, meaning that the attributed features must mirror the input features that influence the output. One recent trend to test faithfulness is to fit a model on designed data with known relevant features and then compare attributions with ground truth input features.This idea assumes that the model learns to use all and only these designed features, for which there is no guarantee. In this paper, we solve this issue by designing the network and manually setting its weights, along with designing data. The setup, AttributionLab, serves as a sanity check for faithfulness: If an attribution method is not faithful in a controlled environment, it can be unreliable in the wild. The environment is also a laboratory for controlled experiments by which we can analyze attribution methods and suggest improvements. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_06514 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | AttributionLab: Faithfulness of Feature Attribution Under Controllable Environments Zhang, Yang Li, Yawei Brown, Hannah Rezaei, Mina Bischl, Bernd Torr, Philip Khakzar, Ashkan Kawaguchi, Kenji Machine Learning Feature attribution explains neural network outputs by identifying relevant input features. The attribution has to be faithful, meaning that the attributed features must mirror the input features that influence the output. One recent trend to test faithfulness is to fit a model on designed data with known relevant features and then compare attributions with ground truth input features.This idea assumes that the model learns to use all and only these designed features, for which there is no guarantee. In this paper, we solve this issue by designing the network and manually setting its weights, along with designing data. The setup, AttributionLab, serves as a sanity check for faithfulness: If an attribution method is not faithful in a controlled environment, it can be unreliable in the wild. The environment is also a laboratory for controlled experiments by which we can analyze attribution methods and suggest improvements. |
| title | AttributionLab: Faithfulness of Feature Attribution Under Controllable Environments |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2310.06514 |