MDD-Net: Multimodal Depression Detection through Mutual Transformer
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913984734560256 |
|---|---|
| author | Haque, Md Rezwanul Islam, Md. Milon Raju, S M Taslim Uddin Altaheri, Hamdi Nassar, Lobna Karray, Fakhri |
| author_facet | Haque, Md Rezwanul Islam, Md. Milon Raju, S M Taslim Uddin Altaheri, Hamdi Nassar, Lobna Karray, Fakhri |
| contents | Depression is a major mental health condition that severely impacts the emotional and physical well-being of individuals. The simple nature of data collection from social media platforms has attracted significant interest in properly utilizing this information for mental health research. A Multimodal Depression Detection Network (MDD-Net), utilizing acoustic and visual data obtained from social media networks, is proposed in this work where mutual transformers are exploited to efficiently extract and fuse multimodal features for efficient depression detection. The MDD-Net consists of four core modules: an acoustic feature extraction module for retrieving relevant acoustic attributes, a visual feature extraction module for extracting significant high-level patterns, a mutual transformer for computing the correlations among the generated features and fusing these features from multiple modalities, and a detection layer for detecting depression using the fused feature representations. The extensive experiments are performed using the multimodal D-Vlog dataset, and the findings reveal that the developed multimodal depression detection network surpasses the state-of-the-art by up to 17.37% for F1-Score, demonstrating the greater performance of the proposed system. The source code is accessible at https://github.com/rezwanh001/Multimodal-Depression-Detection. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_08093 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MDD-Net: Multimodal Depression Detection through Mutual Transformer Haque, Md Rezwanul Islam, Md. Milon Raju, S M Taslim Uddin Altaheri, Hamdi Nassar, Lobna Karray, Fakhri Computer Vision and Pattern Recognition Machine Learning Multimedia Audio and Speech Processing Depression is a major mental health condition that severely impacts the emotional and physical well-being of individuals. The simple nature of data collection from social media platforms has attracted significant interest in properly utilizing this information for mental health research. A Multimodal Depression Detection Network (MDD-Net), utilizing acoustic and visual data obtained from social media networks, is proposed in this work where mutual transformers are exploited to efficiently extract and fuse multimodal features for efficient depression detection. The MDD-Net consists of four core modules: an acoustic feature extraction module for retrieving relevant acoustic attributes, a visual feature extraction module for extracting significant high-level patterns, a mutual transformer for computing the correlations among the generated features and fusing these features from multiple modalities, and a detection layer for detecting depression using the fused feature representations. The extensive experiments are performed using the multimodal D-Vlog dataset, and the findings reveal that the developed multimodal depression detection network surpasses the state-of-the-art by up to 17.37% for F1-Score, demonstrating the greater performance of the proposed system. The source code is accessible at https://github.com/rezwanh001/Multimodal-Depression-Detection. |
| title | MDD-Net: Multimodal Depression Detection through Mutual Transformer |
| topic | Computer Vision and Pattern Recognition Machine Learning Multimedia Audio and Speech Processing |
| url | https://arxiv.org/abs/2508.08093 |