Decouple, Reorganize, and Fuse: A Multimodal Framework for Cancer Survival Prediction
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911463576174592 |
|---|---|
| author | Wang, Huayi Ying, Haochao Xu, Yuyang Qiu, Qibo Zhang, Cheng Chen, Danny Z. Sun, Ying Wu, Jian |
| author_facet | Wang, Huayi Ying, Haochao Xu, Yuyang Qiu, Qibo Zhang, Cheng Chen, Danny Z. Sun, Ying Wu, Jian |
| contents | Cancer survival analysis commonly integrates information across diverse medical modalities to make survival-time predictions. Existing methods primarily focus on extracting different decoupled features of modalities and performing fusion operations such as concatenation, attention, and \revm{Mixture-of-Experts (MoE)-based fusion. However, these methods still face two key challenges: i) Fixed fusion schemes (concatenation and attention) can lead to model over-reliance on predefined feature combinations, limiting the dynamic fusion of decoupled features; ii) in MoE-based fusion methods, each expert network handles separate decoupled features, which limits information interaction among the decoupled features. To address these challenges, we propose a novel Decoupling-Reorganization-Fusion framework (DeReF), which devises a random feature reorganization strategy between modalities decoupling and dynamic MoE fusion modules.Its advantages are: i) it increases the diversity of feature combinations and granularity, enhancing the generalization ability of the subsequent expert networks; ii) it overcomes the problem of information closure and helps expert networks better capture information among decoupled features. Additionally, we incorporate a regional cross-attention network within the modality decoupling module to improve the representation quality of decoupled features. Extensive experimental results on our in-house Liver Cancer (LC) and three widely used TCGA public datasets confirm the effectiveness of our proposed method. Codes are available at https://github.com/ZJUMAI/DeReF. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_18632 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Decouple, Reorganize, and Fuse: A Multimodal Framework for Cancer Survival Prediction Wang, Huayi Ying, Haochao Xu, Yuyang Qiu, Qibo Zhang, Cheng Chen, Danny Z. Sun, Ying Wu, Jian Computer Vision and Pattern Recognition Cancer survival analysis commonly integrates information across diverse medical modalities to make survival-time predictions. Existing methods primarily focus on extracting different decoupled features of modalities and performing fusion operations such as concatenation, attention, and \revm{Mixture-of-Experts (MoE)-based fusion. However, these methods still face two key challenges: i) Fixed fusion schemes (concatenation and attention) can lead to model over-reliance on predefined feature combinations, limiting the dynamic fusion of decoupled features; ii) in MoE-based fusion methods, each expert network handles separate decoupled features, which limits information interaction among the decoupled features. To address these challenges, we propose a novel Decoupling-Reorganization-Fusion framework (DeReF), which devises a random feature reorganization strategy between modalities decoupling and dynamic MoE fusion modules.Its advantages are: i) it increases the diversity of feature combinations and granularity, enhancing the generalization ability of the subsequent expert networks; ii) it overcomes the problem of information closure and helps expert networks better capture information among decoupled features. Additionally, we incorporate a regional cross-attention network within the modality decoupling module to improve the representation quality of decoupled features. Extensive experimental results on our in-house Liver Cancer (LC) and three widely used TCGA public datasets confirm the effectiveness of our proposed method. Codes are available at https://github.com/ZJUMAI/DeReF. |
| title | Decouple, Reorganize, and Fuse: A Multimodal Framework for Cancer Survival Prediction |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2508.18632 |