Retrieval over Classification: Integrating Relation Semantics for Multimodal Relation Extraction
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908558521532416 |
|---|---|
| author | Hei, Lei Liao, Tingjing Pei, Yingxin Qi, Yiyang Wang, Jiaqi Li, Ruiting Ren, Feiliang |
| author_facet | Hei, Lei Liao, Tingjing Pei, Yingxin Qi, Yiyang Wang, Jiaqi Li, Ruiting Ren, Feiliang |
| contents | Relation extraction (RE) aims to identify semantic relations between entities in unstructured text. Although recent work extends traditional RE to multimodal scenarios, most approaches still adopt classification-based paradigms with fused multimodal features, representing relations as discrete labels. This paradigm has two significant limitations: (1) it overlooks structural constraints like entity types and positional cues, and (2) it lacks semantic expressiveness for fine-grained relation understanding. We propose \underline{R}etrieval \underline{O}ver \underline{C}lassification (ROC), a novel framework that reformulates multimodal RE as a retrieval task driven by relation semantics. ROC integrates entity type and positional information through a multimodal encoder, expands relation labels into natural language descriptions using a large language model, and aligns entity-relation pairs via semantic similarity-based contrastive learning. Experiments show that our method achieves state-of-the-art performance on the benchmark datasets MNRE and MORE and exhibits stronger robustness and interpretability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_21151 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Retrieval over Classification: Integrating Relation Semantics for Multimodal Relation Extraction Hei, Lei Liao, Tingjing Pei, Yingxin Qi, Yiyang Wang, Jiaqi Li, Ruiting Ren, Feiliang Computation and Language Information Retrieval Relation extraction (RE) aims to identify semantic relations between entities in unstructured text. Although recent work extends traditional RE to multimodal scenarios, most approaches still adopt classification-based paradigms with fused multimodal features, representing relations as discrete labels. This paradigm has two significant limitations: (1) it overlooks structural constraints like entity types and positional cues, and (2) it lacks semantic expressiveness for fine-grained relation understanding. We propose \underline{R}etrieval \underline{O}ver \underline{C}lassification (ROC), a novel framework that reformulates multimodal RE as a retrieval task driven by relation semantics. ROC integrates entity type and positional information through a multimodal encoder, expands relation labels into natural language descriptions using a large language model, and aligns entity-relation pairs via semantic similarity-based contrastive learning. Experiments show that our method achieves state-of-the-art performance on the benchmark datasets MNRE and MORE and exhibits stronger robustness and interpretability. |
| title | Retrieval over Classification: Integrating Relation Semantics for Multimodal Relation Extraction |
| topic | Computation and Language Information Retrieval |
| url | https://arxiv.org/abs/2509.21151 |