Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.15322 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910838145679360 |
|---|---|
| author | Feng, Bin Ruan, Shulan Yang, Mingzheng Han, Dongxuan Liu, Huijie Zhang, Kai Liu, Qi |
| author_facet | Feng, Bin Ruan, Shulan Yang, Mingzheng Han, Dongxuan Liu, Huijie Zhang, Kai Liu, Qi |
| contents | As more and more internet users post images online to express their daily emotions, image sentiment analysis has attracted increasing attention. Recently, researchers generally tend to design different neural networks to extract visual features from images for sentiment analysis. Despite the significant progress, metadata, the data (e.g., text descriptions and keyword tags) for describing the image, has not been sufficiently explored in this task. In this paper, we propose a novel Metadata Enhanced Transformer for sentiment analysis (SentiFormer) to fuse multiple metadata and the corresponding image into a unified framework. Specifically, we first obtain multiple metadata of the image and unify the representations of diverse data. To adaptively learn the appropriate weights for each metadata, we then design an adaptive relevance learning module to highlight more effective information while suppressing weaker ones. Moreover, we further develop a cross-modal fusion module to fuse the adaptively learned representations and make the final prediction. Extensive experiments on three publicly available datasets demonstrate the superiority and rationality of our proposed method. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_15322 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SentiFormer: Metadata Enhanced Transformer for Image Sentiment Analysis Feng, Bin Ruan, Shulan Yang, Mingzheng Han, Dongxuan Liu, Huijie Zhang, Kai Liu, Qi Computer Vision and Pattern Recognition Artificial Intelligence As more and more internet users post images online to express their daily emotions, image sentiment analysis has attracted increasing attention. Recently, researchers generally tend to design different neural networks to extract visual features from images for sentiment analysis. Despite the significant progress, metadata, the data (e.g., text descriptions and keyword tags) for describing the image, has not been sufficiently explored in this task. In this paper, we propose a novel Metadata Enhanced Transformer for sentiment analysis (SentiFormer) to fuse multiple metadata and the corresponding image into a unified framework. Specifically, we first obtain multiple metadata of the image and unify the representations of diverse data. To adaptively learn the appropriate weights for each metadata, we then design an adaptive relevance learning module to highlight more effective information while suppressing weaker ones. Moreover, we further develop a cross-modal fusion module to fuse the adaptively learned representations and make the final prediction. Extensive experiments on three publicly available datasets demonstrate the superiority and rationality of our proposed method. |
| title | SentiFormer: Metadata Enhanced Transformer for Image Sentiment Analysis |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2502.15322 |