Architecture-Agnostic Modality-Isolated Gated Fusion for Robust Multi-Modal Prostate MRI Segmentation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908974797815808 |
|---|---|
| author | Shu, Yongbo Xie, Wenzhao Yao, Shanhu Xin, Zirui Lei, Luo Chen, Kewen Luo, Aijing |
| author_facet | Shu, Yongbo Xie, Wenzhao Yao, Shanhu Xin, Zirui Lei, Luo Chen, Kewen Luo, Aijing |
| contents | Multi-parametric prostate MRI combines T2-weighted (T2W), apparent diffusion coefficient (ADC), and high b-value diffusion-weighted (HBV) sequences for non-invasive detection of clinically significant prostate cancer. In practice, the diffusion sequences are more frequently subject to acquisition variability, motion, and artifacts than T2W, making robust fusion of these channels the clinically relevant requirement. We propose Modality-Isolated Gated Fusion (MIGF), an architecture-agnostic module that maintains separate modality-specific encoding streams before a learned gating stage, combined with modality dropout training to enforce compensation under incomplete inputs. We benchmark six backbones and assess
MIGF-equipped models under seven missing-modality and artifact scenarios on PI-CAI (1,500 studies, fold-0 split, five seeds). MIGF improved ideal-scenario Ranking Score for UNet, nnUNet, and Mamba by 2.8%, 4.6%, and 13.4%; the best model (MIGFNet-nnUNet, gating + ModDrop, no deep supervision) achieved 0.7304 +/- 0.056. MIGF primarily improved tolerance to HBV/ADC degradation, while missing T2W remained a shared failure mode across all architectures. Mechanistic analysis shows that robustness gains arise from strict modality isolation and dropout-driven compensation rather than adaptive per-sample quality routing: the gate converged to a stable modality prior, and deep supervision was beneficial only for the largest backbone. External evaluation on Prostate158 (n=158) revealed substantial domain shift driven primarily by ADC map incompatibility across institutions; modality-isolation analysis confirmed that removing ADC improved external performance, identifying computed diffusion maps as inherently less portable than raw-acquired sequences. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_10702 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Architecture-Agnostic Modality-Isolated Gated Fusion for Robust Multi-Modal Prostate MRI Segmentation Shu, Yongbo Xie, Wenzhao Yao, Shanhu Xin, Zirui Lei, Luo Chen, Kewen Luo, Aijing Computer Vision and Pattern Recognition Artificial Intelligence Multi-parametric prostate MRI combines T2-weighted (T2W), apparent diffusion coefficient (ADC), and high b-value diffusion-weighted (HBV) sequences for non-invasive detection of clinically significant prostate cancer. In practice, the diffusion sequences are more frequently subject to acquisition variability, motion, and artifacts than T2W, making robust fusion of these channels the clinically relevant requirement. We propose Modality-Isolated Gated Fusion (MIGF), an architecture-agnostic module that maintains separate modality-specific encoding streams before a learned gating stage, combined with modality dropout training to enforce compensation under incomplete inputs. We benchmark six backbones and assess MIGF-equipped models under seven missing-modality and artifact scenarios on PI-CAI (1,500 studies, fold-0 split, five seeds). MIGF improved ideal-scenario Ranking Score for UNet, nnUNet, and Mamba by 2.8%, 4.6%, and 13.4%; the best model (MIGFNet-nnUNet, gating + ModDrop, no deep supervision) achieved 0.7304 +/- 0.056. MIGF primarily improved tolerance to HBV/ADC degradation, while missing T2W remained a shared failure mode across all architectures. Mechanistic analysis shows that robustness gains arise from strict modality isolation and dropout-driven compensation rather than adaptive per-sample quality routing: the gate converged to a stable modality prior, and deep supervision was beneficial only for the largest backbone. External evaluation on Prostate158 (n=158) revealed substantial domain shift driven primarily by ADC map incompatibility across institutions; modality-isolation analysis confirmed that removing ADC improved external performance, identifying computed diffusion maps as inherently less portable than raw-acquired sequences. |
| title | Architecture-Agnostic Modality-Isolated Gated Fusion for Robust Multi-Modal Prostate MRI Segmentation |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2604.10702 |