FSRF: Factorization-guided Semantic Recovery for Incomplete Multimodal Sentiment Analysis
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914100845477888 |
|---|---|
| author | Liu, Ziyang Chu, Pengjunfei Dong, Shuming Zhang, Chen Li, Mingcheng Wang, Jin |
| author_facet | Liu, Ziyang Chu, Pengjunfei Dong, Shuming Zhang, Chen Li, Mingcheng Wang, Jin |
| contents | In recent years, Multimodal Sentiment Analysis (MSA) has become a research hotspot that aims to utilize multimodal data for human sentiment understanding. Previous MSA studies have mainly focused on performing interaction and fusion on complete multimodal data, ignoring the problem of missing modalities in real-world applications due to occlusion, personal privacy constraints, and device malfunctions, resulting in low generalizability.
To this end, we propose a Factorization-guided Semantic Recovery Framework (FSRF) to mitigate the modality missing problem in the MSA task.
Specifically, we propose a de-redundant homo-heterogeneous factorization module that factorizes modality into modality-homogeneous, modality-heterogeneous, and noisy representations and design elaborate constraint paradigms for representation learning.
Furthermore, we design a distribution-aligned self-distillation module that fully recovers the missing semantics by utilizing bidirectional knowledge transfer.
Comprehensive experiments on two datasets indicate that FSRF has a significant performance advantage over previous methods with uncertain missing modalities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_16086 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FSRF: Factorization-guided Semantic Recovery for Incomplete Multimodal Sentiment Analysis Liu, Ziyang Chu, Pengjunfei Dong, Shuming Zhang, Chen Li, Mingcheng Wang, Jin Machine Learning Applications In recent years, Multimodal Sentiment Analysis (MSA) has become a research hotspot that aims to utilize multimodal data for human sentiment understanding. Previous MSA studies have mainly focused on performing interaction and fusion on complete multimodal data, ignoring the problem of missing modalities in real-world applications due to occlusion, personal privacy constraints, and device malfunctions, resulting in low generalizability. To this end, we propose a Factorization-guided Semantic Recovery Framework (FSRF) to mitigate the modality missing problem in the MSA task. Specifically, we propose a de-redundant homo-heterogeneous factorization module that factorizes modality into modality-homogeneous, modality-heterogeneous, and noisy representations and design elaborate constraint paradigms for representation learning. Furthermore, we design a distribution-aligned self-distillation module that fully recovers the missing semantics by utilizing bidirectional knowledge transfer. Comprehensive experiments on two datasets indicate that FSRF has a significant performance advantage over previous methods with uncertain missing modalities. |
| title | FSRF: Factorization-guided Semantic Recovery for Incomplete Multimodal Sentiment Analysis |
| topic | Machine Learning Applications |
| url | https://arxiv.org/abs/2510.16086 |