Not Only the Last-Layer Features for Spurious Correlations: All Layer Deep Feature Reweighting
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866929509840715776 |
|---|---|
| author | Hameed, Humza Wajid Nanfack, Geraldin Belilovsky, Eugene |
| author_facet | Hameed, Humza Wajid Nanfack, Geraldin Belilovsky, Eugene |
| contents | Spurious correlations are a major source of errors for machine learning models, in particular when aiming for group-level fairness. It has been recently shown that a powerful approach to combat spurious correlations is to re-train the last layer on a balanced validation dataset, isolating robust features for the predictor. However, key attributes can sometimes be discarded by neural networks towards the last layer. In this work, we thus consider retraining a classifier on a set of features derived from all layers. We utilize a recently proposed feature selection strategy to select unbiased features from all the layers. We observe this approach gives significant improvements in worst-group accuracy on several standard benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_14637 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Not Only the Last-Layer Features for Spurious Correlations: All Layer Deep Feature Reweighting Hameed, Humza Wajid Nanfack, Geraldin Belilovsky, Eugene Machine Learning Artificial Intelligence Spurious correlations are a major source of errors for machine learning models, in particular when aiming for group-level fairness. It has been recently shown that a powerful approach to combat spurious correlations is to re-train the last layer on a balanced validation dataset, isolating robust features for the predictor. However, key attributes can sometimes be discarded by neural networks towards the last layer. In this work, we thus consider retraining a classifier on a set of features derived from all layers. We utilize a recently proposed feature selection strategy to select unbiased features from all the layers. We observe this approach gives significant improvements in worst-group accuracy on several standard benchmarks. |
| title | Not Only the Last-Layer Features for Spurious Correlations: All Layer Deep Feature Reweighting |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2409.14637 |