Keypoint Counting Classifiers: Turning Vision Transformers into Self-Explainable Models Without Training
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909971224985600 |
|---|---|
| author | Wickstrøm, Kristoffer Dorszewski, Teresa Chen, Siyan Kampffmeyer, Michael Wetzer, Elisabeth Jenssen, Robert |
| author_facet | Wickstrøm, Kristoffer Dorszewski, Teresa Chen, Siyan Kampffmeyer, Michael Wetzer, Elisabeth Jenssen, Robert |
| contents | Current approaches for designing self-explainable models (SEMs) require complicated training procedures and specific architectures which makes them impractical. With the advance of general purpose foundation models based on Vision Transformers (ViTs), this impracticability becomes even more problematic. Therefore, new methods are necessary to provide transparency and reliability to ViT-based foundation models. In this work, we present a new method for turning any well-trained ViT-based model into a SEM without retraining, which we call Keypoint Counting Classifiers (KCCs). Recent works have shown that ViTs can automatically identify matching keypoints between images with high precision, and we build on these results to create an easily interpretable decision process that is inherently visualizable in the input. We perform an extensive evaluation which show that KCCs improve the human-machine communication compared to recent baselines. We believe that KCCs constitute an important step towards making ViT-based foundation models more transparent and reliable. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_17891 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Keypoint Counting Classifiers: Turning Vision Transformers into Self-Explainable Models Without Training Wickstrøm, Kristoffer Dorszewski, Teresa Chen, Siyan Kampffmeyer, Michael Wetzer, Elisabeth Jenssen, Robert Computer Vision and Pattern Recognition Current approaches for designing self-explainable models (SEMs) require complicated training procedures and specific architectures which makes them impractical. With the advance of general purpose foundation models based on Vision Transformers (ViTs), this impracticability becomes even more problematic. Therefore, new methods are necessary to provide transparency and reliability to ViT-based foundation models. In this work, we present a new method for turning any well-trained ViT-based model into a SEM without retraining, which we call Keypoint Counting Classifiers (KCCs). Recent works have shown that ViTs can automatically identify matching keypoints between images with high precision, and we build on these results to create an easily interpretable decision process that is inherently visualizable in the input. We perform an extensive evaluation which show that KCCs improve the human-machine communication compared to recent baselines. We believe that KCCs constitute an important step towards making ViT-based foundation models more transparent and reliable. |
| title | Keypoint Counting Classifiers: Turning Vision Transformers into Self-Explainable Models Without Training |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.17891 |