A Robust Incomplete Multimodal Low-Rank Adaptation Approach for Emotion Recognition
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908871063240704 |
|---|---|
| author | Zhao, Xinkui Shu, Jinsong Wu, Yangyang Cheng, Guanjie Liu, Zihe Wang, Naibo Deng, Shuiguang Xie, Zhongle Yin, Jianwei |
| author_facet | Zhao, Xinkui Shu, Jinsong Wu, Yangyang Cheng, Guanjie Liu, Zihe Wang, Naibo Deng, Shuiguang Xie, Zhongle Yin, Jianwei |
| contents | Multimodal Emotion Recognition (MER) often encounters incomplete multimodality in practical applications due to sensor failures or privacy protection requirements. While existing methods attempt to address various incomplete multimodal scenarios by balancing the training of each modality combination through additional gradients, these approaches face a critical limitation: training gradients from different modality combinations conflict with each other, ultimately degrading the performance of the final prediction model. In this paper, we propose a unimodal decoupled dynamic low-rank adaptation method based on modality combinations, named MCULoRA, which is a novel framework for the parameter-efficient training of incomplete multimodal learning models. MCULoRA consists of two key modules, modality combination aware low-rank adaptation (MCLA) and dynamic parameter fine-tuning (DPFT). The MCLA module effectively decouples the shared information from the distinct characteristics of individual modality combinations. The DPFT module adjusts the training ratio of modality combinations based on the separability of each modality's representation space, optimizing the learning efficiency across different modality combinations. Our extensive experimental evaluation in multiple benchmark datasets demonstrates that MCULoRA substantially outperforms previous incomplete multimodal learning approaches in downstream task accuracy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_11202 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Robust Incomplete Multimodal Low-Rank Adaptation Approach for Emotion Recognition Zhao, Xinkui Shu, Jinsong Wu, Yangyang Cheng, Guanjie Liu, Zihe Wang, Naibo Deng, Shuiguang Xie, Zhongle Yin, Jianwei Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning Multimodal Emotion Recognition (MER) often encounters incomplete multimodality in practical applications due to sensor failures or privacy protection requirements. While existing methods attempt to address various incomplete multimodal scenarios by balancing the training of each modality combination through additional gradients, these approaches face a critical limitation: training gradients from different modality combinations conflict with each other, ultimately degrading the performance of the final prediction model. In this paper, we propose a unimodal decoupled dynamic low-rank adaptation method based on modality combinations, named MCULoRA, which is a novel framework for the parameter-efficient training of incomplete multimodal learning models. MCULoRA consists of two key modules, modality combination aware low-rank adaptation (MCLA) and dynamic parameter fine-tuning (DPFT). The MCLA module effectively decouples the shared information from the distinct characteristics of individual modality combinations. The DPFT module adjusts the training ratio of modality combinations based on the separability of each modality's representation space, optimizing the learning efficiency across different modality combinations. Our extensive experimental evaluation in multiple benchmark datasets demonstrates that MCULoRA substantially outperforms previous incomplete multimodal learning approaches in downstream task accuracy. |
| title | A Robust Incomplete Multimodal Low-Rank Adaptation Approach for Emotion Recognition |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2507.11202 |