Graph-based Interaction Augmentation Network for Robust Multimodal Sentiment Analysis
Fuente:
arXiv
Salvato in:
| Autori principali: | , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913971847561216 |
|---|---|
| author | Zhangfeng, Hu mengxin, Shi |
| author_facet | Zhangfeng, Hu mengxin, Shi |
| contents | The inevitable modality imperfection in real-world scenarios poses significant challenges for Multimodal Sentiment Analysis (MSA). While existing methods tailor reconstruction or joint representation learning strategies to restore missing semantics, they often overlook complex dependencies within and across modalities. Consequently, they fail to fully leverage available modalities to capture complementary semantics. To this end, this paper proposes a novel graph-based framework to exploit both intra- and inter-modality interactions, enabling imperfect samples to derive missing semantics from complementary parts for robust MSA. Specifically, we first devise a learnable hypergraph to model intra-modality temporal dependencies to exploit contextual information within each modality. Then, a directed graph is employed to explore inter-modality correlations based on attention mechanism, capturing complementary information across different modalities. Finally, the knowledge from perfect samples is integrated to supervise our interaction processes, guiding the model toward learning reliable and robust joint representations. Extensive experiments on MOSI and MOSEI datasets demonstrate the effectiveness of our method. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_01168 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Graph-based Interaction Augmentation Network for Robust Multimodal Sentiment Analysis Zhangfeng, Hu mengxin, Shi Multimedia The inevitable modality imperfection in real-world scenarios poses significant challenges for Multimodal Sentiment Analysis (MSA). While existing methods tailor reconstruction or joint representation learning strategies to restore missing semantics, they often overlook complex dependencies within and across modalities. Consequently, they fail to fully leverage available modalities to capture complementary semantics. To this end, this paper proposes a novel graph-based framework to exploit both intra- and inter-modality interactions, enabling imperfect samples to derive missing semantics from complementary parts for robust MSA. Specifically, we first devise a learnable hypergraph to model intra-modality temporal dependencies to exploit contextual information within each modality. Then, a directed graph is employed to explore inter-modality correlations based on attention mechanism, capturing complementary information across different modalities. Finally, the knowledge from perfect samples is integrated to supervise our interaction processes, guiding the model toward learning reliable and robust joint representations. Extensive experiments on MOSI and MOSEI datasets demonstrate the effectiveness of our method. |
| title | Graph-based Interaction Augmentation Network for Robust Multimodal Sentiment Analysis |
| topic | Multimedia |
| url | https://arxiv.org/abs/2508.01168 |