X Hacking: The Threat of Misguided AutoML
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911056323936256 |
|---|---|
| author | Sharma, Rahul Redyuk, Sergey Mukherjee, Sumantrak Šipka, Andrea Hüllermeier, Eyke Vollmer, Sebastian Selby, David |
| author_facet | Sharma, Rahul Redyuk, Sergey Mukherjee, Sumantrak Šipka, Andrea Hüllermeier, Eyke Vollmer, Sebastian Selby, David |
| contents | Explainable AI (XAI) and interpretable machine learning methods help to build trust in model predictions and derived insights, yet also present a perverse incentive for analysts to manipulate XAI metrics to support pre-specified conclusions. This paper introduces the concept of X-hacking, a form of p-hacking applied to XAI metrics such as SHAP values. We show how easily an automated machine learning pipeline can be adapted to exploit model multiplicity at scale: searching a Rashomon set of 'defensible' models with similar predictive performance to find a desired explanation. We formulate the trade-off between explanation and accuracy as a multi-objective optimisation problem, and illustrate empirically on familiar real-world datasets that, on average, Bayesian optimisation accelerates X-hacking 3-fold for features susceptible to it, versus random sampling. We show the vulnerability of a dataset to X-hacking can be determined by information redundancy among features. Finally, we suggest possible methods for detection and prevention, and discuss ethical implications for the credibility and reproducibility of XAI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_08513 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | X Hacking: The Threat of Misguided AutoML Sharma, Rahul Redyuk, Sergey Mukherjee, Sumantrak Šipka, Andrea Hüllermeier, Eyke Vollmer, Sebastian Selby, David Machine Learning Cryptography and Security Explainable AI (XAI) and interpretable machine learning methods help to build trust in model predictions and derived insights, yet also present a perverse incentive for analysts to manipulate XAI metrics to support pre-specified conclusions. This paper introduces the concept of X-hacking, a form of p-hacking applied to XAI metrics such as SHAP values. We show how easily an automated machine learning pipeline can be adapted to exploit model multiplicity at scale: searching a Rashomon set of 'defensible' models with similar predictive performance to find a desired explanation. We formulate the trade-off between explanation and accuracy as a multi-objective optimisation problem, and illustrate empirically on familiar real-world datasets that, on average, Bayesian optimisation accelerates X-hacking 3-fold for features susceptible to it, versus random sampling. We show the vulnerability of a dataset to X-hacking can be determined by information redundancy among features. Finally, we suggest possible methods for detection and prevention, and discuss ethical implications for the credibility and reproducibility of XAI. |
| title | X Hacking: The Threat of Misguided AutoML |
| topic | Machine Learning Cryptography and Security |
| url | https://arxiv.org/abs/2401.08513 |