Multimodal Magic Elevating Depression Detection with a Fusion of Text and Audio Intelligence
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917907205718016 |
|---|---|
| author | Gan, Lindy Huang, Yifan Gao, Xiaoyang Tan, Jiaming Zhao, Fujun Yang, Tao |
| author_facet | Gan, Lindy Huang, Yifan Gao, Xiaoyang Tan, Jiaming Zhao, Fujun Yang, Tao |
| contents | This study proposes an innovative multimodal fusion model based on a teacher-student architecture to enhance the accuracy of depression classification. Our designed model addresses the limitations of traditional methods in feature fusion and modality weight allocation by introducing multi-head attention mechanisms and weighted multimodal transfer learning. Leveraging the DAIC-WOZ dataset, the student fusion model, guided by textual and auditory teacher models, achieves significant improvements in classification accuracy. Ablation experiments demonstrate that the proposed model attains an F1 score of 99. 1% on the test set, significantly outperforming unimodal and conventional approaches. Our method effectively captures the complementarity between textual and audio features while dynamically adjusting the contributions of the teacher models to enhance generalization capabilities. The experimental results highlight the robustness and adaptability of the proposed framework in handling complex multimodal data. This research provides a novel technical framework for multimodal large model learning in depression analysis, offering new insights into addressing the limitations of existing methods in modality fusion and feature extraction. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_16813 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Multimodal Magic Elevating Depression Detection with a Fusion of Text and Audio Intelligence Gan, Lindy Huang, Yifan Gao, Xiaoyang Tan, Jiaming Zhao, Fujun Yang, Tao Computation and Language Sound Audio and Speech Processing This study proposes an innovative multimodal fusion model based on a teacher-student architecture to enhance the accuracy of depression classification. Our designed model addresses the limitations of traditional methods in feature fusion and modality weight allocation by introducing multi-head attention mechanisms and weighted multimodal transfer learning. Leveraging the DAIC-WOZ dataset, the student fusion model, guided by textual and auditory teacher models, achieves significant improvements in classification accuracy. Ablation experiments demonstrate that the proposed model attains an F1 score of 99. 1% on the test set, significantly outperforming unimodal and conventional approaches. Our method effectively captures the complementarity between textual and audio features while dynamically adjusting the contributions of the teacher models to enhance generalization capabilities. The experimental results highlight the robustness and adaptability of the proposed framework in handling complex multimodal data. This research provides a novel technical framework for multimodal large model learning in depression analysis, offering new insights into addressing the limitations of existing methods in modality fusion and feature extraction. |
| title | Multimodal Magic Elevating Depression Detection with a Fusion of Text and Audio Intelligence |
| topic | Computation and Language Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2501.16813 |