VIViT: Variable-Input Vision Transformer Framework for 3D MR Image Segmentation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912429529628672 |
|---|---|
| author | Das, Badhan Kumar Singh, Ajay Zhao, Gengyan Liu, Han Re, Thomas J. Comaniciu, Dorin Gibson, Eli Maier, Andreas |
| author_facet | Das, Badhan Kumar Singh, Ajay Zhao, Gengyan Liu, Han Re, Thomas J. Comaniciu, Dorin Gibson, Eli Maier, Andreas |
| contents | Self-supervised pretrain techniques have been widely used to improve the downstream tasks' performance. However, real-world magnetic resonance (MR) studies usually consist of different sets of contrasts due to different acquisition protocols, which poses challenges for the current deep learning methods on large-scale pretrain and different downstream tasks with different input requirements, since these methods typically require a fixed set of input modalities or, contrasts. To address this challenge, we propose variable-input ViT (VIViT), a transformer-based framework designed for self-supervised pretraining and segmentation finetuning for variable contrasts in each study. With this ability, our approach can maximize the data availability in pretrain, and can transfer the learned knowledge from pretrain to downstream tasks despite variations in input requirements. We validate our method on brain infarct and brain tumor segmentation, where our method outperforms current CNN and ViT-based models with a mean Dice score of 0.624 and 0.883 respectively. These results highlight the efficacy of our design for better adaptability and performance on tasks with real-world heterogeneous MR data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_08693 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VIViT: Variable-Input Vision Transformer Framework for 3D MR Image Segmentation Das, Badhan Kumar Singh, Ajay Zhao, Gengyan Liu, Han Re, Thomas J. Comaniciu, Dorin Gibson, Eli Maier, Andreas Image and Video Processing Computer Vision and Pattern Recognition Self-supervised pretrain techniques have been widely used to improve the downstream tasks' performance. However, real-world magnetic resonance (MR) studies usually consist of different sets of contrasts due to different acquisition protocols, which poses challenges for the current deep learning methods on large-scale pretrain and different downstream tasks with different input requirements, since these methods typically require a fixed set of input modalities or, contrasts. To address this challenge, we propose variable-input ViT (VIViT), a transformer-based framework designed for self-supervised pretraining and segmentation finetuning for variable contrasts in each study. With this ability, our approach can maximize the data availability in pretrain, and can transfer the learned knowledge from pretrain to downstream tasks despite variations in input requirements. We validate our method on brain infarct and brain tumor segmentation, where our method outperforms current CNN and ViT-based models with a mean Dice score of 0.624 and 0.883 respectively. These results highlight the efficacy of our design for better adaptability and performance on tasks with real-world heterogeneous MR data. |
| title | VIViT: Variable-Input Vision Transformer Framework for 3D MR Image Segmentation |
| topic | Image and Video Processing Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.08693 |