DynCIM: Dynamic Curriculum for Imbalanced Multimodal Learning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917046599548928 |
|---|---|
| author | Qian, Chengxuan Han, Kai Liu, Jiaxin Yuan, Zhenlong Zhu, Zhengzhong Wang, Jingchao Lyu, Chongwen Chen, Jun Liu, Zhe |
| author_facet | Qian, Chengxuan Han, Kai Liu, Jiaxin Yuan, Zhenlong Zhu, Zhengzhong Wang, Jingchao Lyu, Chongwen Chen, Jun Liu, Zhe |
| contents | Multimodal learning integrates complementary information from diverse modalities to enhance the decision-making process. However, the potential of multimodal collaboration remains under-exploited due to disparities in data quality and modality representation capabilities. To address this, we introduce DynCIM, a novel dynamic curriculum learning framework designed to quantify the inherent imbalances from both sample and modality perspectives. DynCIM employs a sample-level curriculum to dynamically assess each sample's difficulty according to prediction deviation, consistency, and stability, while a modality-level curriculum measures modality contributions from global and local. Furthermore, a gating-based dynamic fusion mechanism is introduced to adaptively adjust modality contributions, minimizing redundancy and optimizing fusion effectiveness. Extensive experiments on six multimodal benchmarking datasets, spanning both bimodal and trimodal scenarios, demonstrate that DynCIM consistently outperforms state-of-the-art methods. Our approach effectively mitigates modality and sample imbalances while enhancing adaptability and robustness in multimodal learning tasks. Our code is available at https://github.com/Raymond-Qiancx/DynCIM. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_06456 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DynCIM: Dynamic Curriculum for Imbalanced Multimodal Learning Qian, Chengxuan Han, Kai Liu, Jiaxin Yuan, Zhenlong Zhu, Zhengzhong Wang, Jingchao Lyu, Chongwen Chen, Jun Liu, Zhe Computer Vision and Pattern Recognition Multimodal learning integrates complementary information from diverse modalities to enhance the decision-making process. However, the potential of multimodal collaboration remains under-exploited due to disparities in data quality and modality representation capabilities. To address this, we introduce DynCIM, a novel dynamic curriculum learning framework designed to quantify the inherent imbalances from both sample and modality perspectives. DynCIM employs a sample-level curriculum to dynamically assess each sample's difficulty according to prediction deviation, consistency, and stability, while a modality-level curriculum measures modality contributions from global and local. Furthermore, a gating-based dynamic fusion mechanism is introduced to adaptively adjust modality contributions, minimizing redundancy and optimizing fusion effectiveness. Extensive experiments on six multimodal benchmarking datasets, spanning both bimodal and trimodal scenarios, demonstrate that DynCIM consistently outperforms state-of-the-art methods. Our approach effectively mitigates modality and sample imbalances while enhancing adaptability and robustness in multimodal learning tasks. Our code is available at https://github.com/Raymond-Qiancx/DynCIM. |
| title | DynCIM: Dynamic Curriculum for Imbalanced Multimodal Learning |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2503.06456 |