Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.07943 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912269566214144 |
|---|---|
| author | Zhao, Taoxu Li, Meisi Chen, Kehao Wang, Liye Zhou, Xucheng Chaturvedi, Kunal Prasad, Mukesh Anaissi, Ali Braytee, Ali |
| author_facet | Zhao, Taoxu Li, Meisi Chen, Kehao Wang, Liye Zhou, Xucheng Chaturvedi, Kunal Prasad, Mukesh Anaissi, Ali Braytee, Ali |
| contents | Multimodal sentiment analysis enhances conventional sentiment analysis, which traditionally relies solely on text, by incorporating information from different modalities such as images, text, and audio. This paper proposes a novel multimodal sentiment analysis architecture that integrates text and image data to provide a more comprehensive understanding of sentiments. For text feature extraction, we utilize BERT, a natural language processing model. For image feature extraction, we employ DINOv2, a vision-transformer-based model. The textual and visual latent features are integrated using proposed fusion techniques, namely the Basic Fusion Model, Self Attention Fusion Model, and Dual Attention Fusion Model. Experiments on three datasets, Memotion 7k dataset, MVSA single dataset, and MVSA multi dataset, demonstrate the viability and practicality of the proposed multimodal architecture. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_07943 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Enhancing Sentiment Analysis through Multimodal Fusion: A BERT-DINOv2 Approach Zhao, Taoxu Li, Meisi Chen, Kehao Wang, Liye Zhou, Xucheng Chaturvedi, Kunal Prasad, Mukesh Anaissi, Ali Braytee, Ali Computer Vision and Pattern Recognition Computation and Language Multimodal sentiment analysis enhances conventional sentiment analysis, which traditionally relies solely on text, by incorporating information from different modalities such as images, text, and audio. This paper proposes a novel multimodal sentiment analysis architecture that integrates text and image data to provide a more comprehensive understanding of sentiments. For text feature extraction, we utilize BERT, a natural language processing model. For image feature extraction, we employ DINOv2, a vision-transformer-based model. The textual and visual latent features are integrated using proposed fusion techniques, namely the Basic Fusion Model, Self Attention Fusion Model, and Dual Attention Fusion Model. Experiments on three datasets, Memotion 7k dataset, MVSA single dataset, and MVSA multi dataset, demonstrate the viability and practicality of the proposed multimodal architecture. |
| title | Enhancing Sentiment Analysis through Multimodal Fusion: A BERT-DINOv2 Approach |
| topic | Computer Vision and Pattern Recognition Computation and Language |
| url | https://arxiv.org/abs/2503.07943 |