Learning Brain Tumor Representation in 3D High-Resolution MR Images via Interpretable State Space Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913498400817152 |
|---|---|
| author | Hu, Qingqiao Zhang, Daoan Luo, Jiebo Gong, Zhenyu Wiestler, Benedikt Zhang, Jianguo Li, Hongwei Bran |
| author_facet | Hu, Qingqiao Zhang, Daoan Luo, Jiebo Gong, Zhenyu Wiestler, Benedikt Zhang, Jianguo Li, Hongwei Bran |
| contents | Learning meaningful and interpretable representations from high-dimensional volumetric magnetic resonance (MR) images is essential for advancing personalized medicine. While Vision Transformers (ViTs) have shown promise in handling image data, their application to 3D multi-contrast MR images faces challenges due to computational complexity and interpretability. To address this, we propose a novel state-space-model (SSM)-based masked autoencoder which scales ViT-like models to handle high-resolution data effectively while also enhancing the interpretability of learned representations. We propose a latent-to-spatial mapping technique that enables direct visualization of how latent features correspond to specific regions in the input volumes in the context of SSM. We validate our method on two key neuro-oncology tasks: identification of isocitrate dehydrogenase mutation status and 1p/19q co-deletion classification, achieving state-of-the-art accuracy. Our results highlight the potential of SSM-based self-supervised learning to transform radiomics analysis by combining efficiency and interpretability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_07746 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Learning Brain Tumor Representation in 3D High-Resolution MR Images via Interpretable State Space Models Hu, Qingqiao Zhang, Daoan Luo, Jiebo Gong, Zhenyu Wiestler, Benedikt Zhang, Jianguo Li, Hongwei Bran Computer Vision and Pattern Recognition Learning meaningful and interpretable representations from high-dimensional volumetric magnetic resonance (MR) images is essential for advancing personalized medicine. While Vision Transformers (ViTs) have shown promise in handling image data, their application to 3D multi-contrast MR images faces challenges due to computational complexity and interpretability. To address this, we propose a novel state-space-model (SSM)-based masked autoencoder which scales ViT-like models to handle high-resolution data effectively while also enhancing the interpretability of learned representations. We propose a latent-to-spatial mapping technique that enables direct visualization of how latent features correspond to specific regions in the input volumes in the context of SSM. We validate our method on two key neuro-oncology tasks: identification of isocitrate dehydrogenase mutation status and 1p/19q co-deletion classification, achieving state-of-the-art accuracy. Our results highlight the potential of SSM-based self-supervised learning to transform radiomics analysis by combining efficiency and interpretability. |
| title | Learning Brain Tumor Representation in 3D High-Resolution MR Images via Interpretable State Space Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2409.07746 |