Relationship Analysis of Image-Text Pair in SNS Posts
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918029192855552 |
|---|---|
| author | Nabeoka, Takuto Duan, Yijun Ma, Qiang |
| author_facet | Nabeoka, Takuto Duan, Yijun Ma, Qiang |
| contents | Social networking services (SNS) contain vast amounts of image-text posts, necessitating effective analysis of their relationships for improved information retrieval. This study addresses the classification of image-text pairs in SNS, overcoming prior limitations in distinguishing relationships beyond similarity. We propose a graph-based method to classify image-text pairs into similar and complementary relationships. Our approach first embeds images and text using CLIP, followed by clustering. Next, we construct an Image-Text Relationship Clustering Line Graph (ITRC-Line Graph), where clusters serve as nodes. Finally, edges and nodes are swapped in a pseudo-graph representation. A Graph Convolutional Network (GCN) then learns node and edge representations, which are fused with the original embeddings for final classification. Experimental results on a publicly available dataset demonstrate the effectiveness of our method. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_15629 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Relationship Analysis of Image-Text Pair in SNS Posts Nabeoka, Takuto Duan, Yijun Ma, Qiang Multimedia Social networking services (SNS) contain vast amounts of image-text posts, necessitating effective analysis of their relationships for improved information retrieval. This study addresses the classification of image-text pairs in SNS, overcoming prior limitations in distinguishing relationships beyond similarity. We propose a graph-based method to classify image-text pairs into similar and complementary relationships. Our approach first embeds images and text using CLIP, followed by clustering. Next, we construct an Image-Text Relationship Clustering Line Graph (ITRC-Line Graph), where clusters serve as nodes. Finally, edges and nodes are swapped in a pseudo-graph representation. A Graph Convolutional Network (GCN) then learns node and edge representations, which are fused with the original embeddings for final classification. Experimental results on a publicly available dataset demonstrate the effectiveness of our method. |
| title | Relationship Analysis of Image-Text Pair in SNS Posts |
| topic | Multimedia |
| url | https://arxiv.org/abs/2505.15629 |