Removing Spurious Concepts from Neural Network Representations via Joint Subspace Estimation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911964401238016 |
|---|---|
| author | Holstege, Floris Wouters, Bram van Giersbergen, Noud Diks, Cees |
| author_facet | Holstege, Floris Wouters, Bram van Giersbergen, Noud Diks, Cees |
| contents | Out-of-distribution generalization in neural networks is often hampered by spurious correlations. A common strategy is to mitigate this by removing spurious concepts from the neural network representation of the data. Existing concept-removal methods tend to be overzealous by inadvertently eliminating features associated with the main task of the model, thereby harming model performance. We propose an iterative algorithm that separates spurious from main-task concepts by jointly identifying two low-dimensional orthogonal subspaces in the neural network representation. We evaluate the algorithm on benchmark datasets for computer vision (Waterbirds, CelebA) and natural language processing (MultiNLI), and show that it outperforms existing concept removal methods |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_11991 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Removing Spurious Concepts from Neural Network Representations via Joint Subspace Estimation Holstege, Floris Wouters, Bram van Giersbergen, Noud Diks, Cees Machine Learning Out-of-distribution generalization in neural networks is often hampered by spurious correlations. A common strategy is to mitigate this by removing spurious concepts from the neural network representation of the data. Existing concept-removal methods tend to be overzealous by inadvertently eliminating features associated with the main task of the model, thereby harming model performance. We propose an iterative algorithm that separates spurious from main-task concepts by jointly identifying two low-dimensional orthogonal subspaces in the neural network representation. We evaluate the algorithm on benchmark datasets for computer vision (Waterbirds, CelebA) and natural language processing (MultiNLI), and show that it outperforms existing concept removal methods |
| title | Removing Spurious Concepts from Neural Network Representations via Joint Subspace Estimation |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2310.11991 |