Pretrained ViTs Yield Versatile Representations For Medical Images
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909390111506432 |
|---|---|
| author | Matsoukas, Christos Haslum, Johan Fredin Sorkhei, Moein Söderberg, Magnus Smith, Kevin |
| author_facet | Matsoukas, Christos Haslum, Johan Fredin Sorkhei, Moein Söderberg, Magnus Smith, Kevin |
| contents | Convolutional Neural Networks (CNNs) have reigned for a decade as the de facto approach to automated medical image diagnosis, pushing the state-of-the-art in classification, detection and segmentation tasks. Over the last years, vision transformers (ViTs) have appeared as a competitive alternative to CNNs, yielding impressive levels of performance in the natural image domain, while possessing several interesting properties that could prove beneficial for medical imaging tasks. In this work, we explore the benefits and drawbacks of transformer-based models for medical image classification. We conduct a series of experiments on several standard 2D medical image benchmark datasets and tasks. Our findings show that, while CNNs perform better if trained from scratch, off-the-shelf vision transformers can perform on par with CNNs when pretrained on ImageNet, both in a supervised and self-supervised setting, rendering them as a viable alternative to CNNs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2303_07034 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Pretrained ViTs Yield Versatile Representations For Medical Images Matsoukas, Christos Haslum, Johan Fredin Sorkhei, Moein Söderberg, Magnus Smith, Kevin Computer Vision and Pattern Recognition Convolutional Neural Networks (CNNs) have reigned for a decade as the de facto approach to automated medical image diagnosis, pushing the state-of-the-art in classification, detection and segmentation tasks. Over the last years, vision transformers (ViTs) have appeared as a competitive alternative to CNNs, yielding impressive levels of performance in the natural image domain, while possessing several interesting properties that could prove beneficial for medical imaging tasks. In this work, we explore the benefits and drawbacks of transformer-based models for medical image classification. We conduct a series of experiments on several standard 2D medical image benchmark datasets and tasks. Our findings show that, while CNNs perform better if trained from scratch, off-the-shelf vision transformers can perform on par with CNNs when pretrained on ImageNet, both in a supervised and self-supervised setting, rendering them as a viable alternative to CNNs. |
| title | Pretrained ViTs Yield Versatile Representations For Medical Images |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2303.07034 |