Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2401.06122 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918295719903232 |
|---|---|
| author | Bareeva, Dilyara Höhne, Marina M. -C. Warnecke, Alexander Pirch, Lukas Müller, Klaus-Robert Rieck, Konrad Lapuschkin, Sebastian Bykov, Kirill |
| author_facet | Bareeva, Dilyara Höhne, Marina M. -C. Warnecke, Alexander Pirch, Lukas Müller, Klaus-Robert Rieck, Konrad Lapuschkin, Sebastian Bykov, Kirill |
| contents | Feature Visualization (FV) is a widely used technique for interpreting concepts learned by Deep Neural Networks (DNNs), which synthesizes input patterns that maximally activate a given feature. Despite its popularity, the trustworthiness of FV explanations has received limited attention. We introduce Gradient Slingshots, a novel method that enables FV manipulation without modifying model architecture or significantly degrading performance. By shaping new trajectories in off-distribution regions of a feature's activation landscape, we coerce the optimization process to converge to a predefined visualization. We evaluate our approach on several DNN architectures, demonstrating its ability to replace faithful FVs with arbitrary targets. These results expose a critical vulnerability: auditors relying solely on FV may accept entirely fabricated explanations. To mitigate this risk, we propose a straightforward defense and quantitatively demonstrate its effectiveness. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_06122 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Manipulating Feature Visualizations with Gradient Slingshots Bareeva, Dilyara Höhne, Marina M. -C. Warnecke, Alexander Pirch, Lukas Müller, Klaus-Robert Rieck, Konrad Lapuschkin, Sebastian Bykov, Kirill Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Feature Visualization (FV) is a widely used technique for interpreting concepts learned by Deep Neural Networks (DNNs), which synthesizes input patterns that maximally activate a given feature. Despite its popularity, the trustworthiness of FV explanations has received limited attention. We introduce Gradient Slingshots, a novel method that enables FV manipulation without modifying model architecture or significantly degrading performance. By shaping new trajectories in off-distribution regions of a feature's activation landscape, we coerce the optimization process to converge to a predefined visualization. We evaluate our approach on several DNN architectures, demonstrating its ability to replace faithful FVs with arbitrary targets. These results expose a critical vulnerability: auditors relying solely on FV may accept entirely fabricated explanations. To mitigate this risk, we propose a straightforward defense and quantitatively demonstrate its effectiveness. |
| title | Manipulating Feature Visualizations with Gradient Slingshots |
| topic | Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2401.06122 |