Incomplete Modality Disentangled Representation for Ophthalmic Disease Grading and Diagnosis
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910831742025728 |
|---|---|
| author | Liu, Chengzhi Huang, Zile Chen, Zhe Tang, Feilong Tian, Yu Xu, Zhongxing Luo, Zihong Zheng, Yalin Meng, Yanda |
| author_facet | Liu, Chengzhi Huang, Zile Chen, Zhe Tang, Feilong Tian, Yu Xu, Zhongxing Luo, Zihong Zheng, Yalin Meng, Yanda |
| contents | Ophthalmologists typically require multimodal data sources to improve diagnostic accuracy in clinical decisions. However, due to medical device shortages, low-quality data and data privacy concerns, missing data modalities are common in real-world scenarios. Existing deep learning methods tend to address it by learning an implicit latent subspace representation for different modality combinations. We identify two significant limitations of these methods: (1) implicit representation constraints that hinder the model's ability to capture modality-specific information and (2) modality heterogeneity, causing distribution gaps and redundancy in feature representations. To address these, we propose an Incomplete Modality Disentangled Representation (IMDR) strategy, which disentangles features into explicit independent modal-common and modal-specific features by guidance of mutual information, distilling informative knowledge and enabling it to reconstruct valuable missing semantics and produce robust multimodal representations. Furthermore, we introduce a joint proxy learning module that assists IMDR in eliminating intra-modality redundancy by exploiting the extracted proxies from each class. Experiments on four ophthalmology multimodal datasets demonstrate that the proposed IMDR outperforms the state-of-the-art methods significantly. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_11724 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Incomplete Modality Disentangled Representation for Ophthalmic Disease Grading and Diagnosis Liu, Chengzhi Huang, Zile Chen, Zhe Tang, Feilong Tian, Yu Xu, Zhongxing Luo, Zihong Zheng, Yalin Meng, Yanda Computer Vision and Pattern Recognition Ophthalmologists typically require multimodal data sources to improve diagnostic accuracy in clinical decisions. However, due to medical device shortages, low-quality data and data privacy concerns, missing data modalities are common in real-world scenarios. Existing deep learning methods tend to address it by learning an implicit latent subspace representation for different modality combinations. We identify two significant limitations of these methods: (1) implicit representation constraints that hinder the model's ability to capture modality-specific information and (2) modality heterogeneity, causing distribution gaps and redundancy in feature representations. To address these, we propose an Incomplete Modality Disentangled Representation (IMDR) strategy, which disentangles features into explicit independent modal-common and modal-specific features by guidance of mutual information, distilling informative knowledge and enabling it to reconstruct valuable missing semantics and produce robust multimodal representations. Furthermore, we introduce a joint proxy learning module that assists IMDR in eliminating intra-modality redundancy by exploiting the extracted proxies from each class. Experiments on four ophthalmology multimodal datasets demonstrate that the proposed IMDR outperforms the state-of-the-art methods significantly. |
| title | Incomplete Modality Disentangled Representation for Ophthalmic Disease Grading and Diagnosis |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2502.11724 |