Efficient Unsupervised Shortcut Learning Detection and Mitigation in Transformers
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909649614143488 |
|---|---|
| author | Kuhn, Lukas Sadiya, Sari Schlotterer, Jorg Buettner, Florian Seifert, Christin Roig, Gemma |
| author_facet | Kuhn, Lukas Sadiya, Sari Schlotterer, Jorg Buettner, Florian Seifert, Christin Roig, Gemma |
| contents | Shortcut learning, i.e., a model's reliance on undesired features not directly relevant to the task, is a major challenge that severely limits the applications of machine learning algorithms, particularly when deploying them to assist in making sensitive decisions, such as in medical diagnostics. In this work, we leverage recent advancements in machine learning to create an unsupervised framework that is capable of both detecting and mitigating shortcut learning in transformers. We validate our method on multiple datasets. Results demonstrate that our framework significantly improves both worst-group accuracy (samples misclassified due to shortcuts) and average accuracy, while minimizing human annotation effort. Moreover, we demonstrate that the detected shortcuts are meaningful and informative to human experts, and that our framework is computationally efficient, allowing it to be run on consumer hardware. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_00942 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Efficient Unsupervised Shortcut Learning Detection and Mitigation in Transformers Kuhn, Lukas Sadiya, Sari Schlotterer, Jorg Buettner, Florian Seifert, Christin Roig, Gemma Machine Learning Computer Vision and Pattern Recognition Shortcut learning, i.e., a model's reliance on undesired features not directly relevant to the task, is a major challenge that severely limits the applications of machine learning algorithms, particularly when deploying them to assist in making sensitive decisions, such as in medical diagnostics. In this work, we leverage recent advancements in machine learning to create an unsupervised framework that is capable of both detecting and mitigating shortcut learning in transformers. We validate our method on multiple datasets. Results demonstrate that our framework significantly improves both worst-group accuracy (samples misclassified due to shortcuts) and average accuracy, while minimizing human annotation effort. Moreover, we demonstrate that the detected shortcuts are meaningful and informative to human experts, and that our framework is computationally efficient, allowing it to be run on consumer hardware. |
| title | Efficient Unsupervised Shortcut Learning Detection and Mitigation in Transformers |
| topic | Machine Learning Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2501.00942 |