Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2401.08868 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914645009235968 |
|---|---|
| author | Tran, Manuel Lahiani, Amal Cid, Yashin Dicente Boxberg, Melanie Lienemann, Peter Matek, Christian Wagner, Sophia J. Theis, Fabian J. Klaiman, Eldad Peng, Tingying |
| author_facet | Tran, Manuel Lahiani, Amal Cid, Yashin Dicente Boxberg, Melanie Lienemann, Peter Matek, Christian Wagner, Sophia J. Theis, Fabian J. Klaiman, Eldad Peng, Tingying |
| contents | Vision Transformers (ViTs) and Swin Transformers (Swin) are currently state-of-the-art in computational pathology. However, domain experts are still reluctant to use these models due to their lack of interpretability. This is not surprising, as critical decisions need to be transparent and understandable. The most common approach to understanding transformers is to visualize their attention. However, attention maps of ViTs are often fragmented, leading to unsatisfactory explanations. Here, we introduce a novel architecture called the B-cos Vision Transformer (BvT) that is designed to be more interpretable. It replaces all linear transformations with the B-cos transform to promote weight-input alignment. In a blinded study, medical experts clearly ranked BvTs above ViTs, suggesting that our network is better at capturing biomedically relevant structures. This is also true for the B-cos Swin Transformer (Bwin). Compared to the Swin Transformer, it even improves the F1-score by up to 4.7% on two public datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_08868 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | B-Cos Aligned Transformers Learn Human-Interpretable Features Tran, Manuel Lahiani, Amal Cid, Yashin Dicente Boxberg, Melanie Lienemann, Peter Matek, Christian Wagner, Sophia J. Theis, Fabian J. Klaiman, Eldad Peng, Tingying Computer Vision and Pattern Recognition Vision Transformers (ViTs) and Swin Transformers (Swin) are currently state-of-the-art in computational pathology. However, domain experts are still reluctant to use these models due to their lack of interpretability. This is not surprising, as critical decisions need to be transparent and understandable. The most common approach to understanding transformers is to visualize their attention. However, attention maps of ViTs are often fragmented, leading to unsatisfactory explanations. Here, we introduce a novel architecture called the B-cos Vision Transformer (BvT) that is designed to be more interpretable. It replaces all linear transformations with the B-cos transform to promote weight-input alignment. In a blinded study, medical experts clearly ranked BvTs above ViTs, suggesting that our network is better at capturing biomedically relevant structures. This is also true for the B-cos Swin Transformer (Bwin). Compared to the Swin Transformer, it even improves the F1-score by up to 4.7% on two public datasets. |
| title | B-Cos Aligned Transformers Learn Human-Interpretable Features |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2401.08868 |