Enhancing Image-Text Matching with Adaptive Feature Aggregation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910300779839488 |
|---|---|
| author | Wang, Zuhui Yin, Yunting Ramakrishnan, I. V. |
| author_facet | Wang, Zuhui Yin, Yunting Ramakrishnan, I. V. |
| contents | Image-text matching aims to find matched cross-modal pairs accurately. While current methods often rely on projecting cross-modal features into a common embedding space, they frequently suffer from imbalanced feature representations across different modalities, leading to unreliable retrieval results. To address these limitations, we introduce a novel Feature Enhancement Module that adaptively aggregates single-modal features for more balanced and robust image-text retrieval. Additionally, we propose a new loss function that overcomes the shortcomings of original triplet ranking loss, thereby significantly improving retrieval performance. The proposed model has been evaluated on two public datasets and achieves competitive retrieval performance when compared with several state-of-the-art models. Implementation codes can be found here. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_09725 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Enhancing Image-Text Matching with Adaptive Feature Aggregation Wang, Zuhui Yin, Yunting Ramakrishnan, I. V. Information Retrieval Multimedia Image-text matching aims to find matched cross-modal pairs accurately. While current methods often rely on projecting cross-modal features into a common embedding space, they frequently suffer from imbalanced feature representations across different modalities, leading to unreliable retrieval results. To address these limitations, we introduce a novel Feature Enhancement Module that adaptively aggregates single-modal features for more balanced and robust image-text retrieval. Additionally, we propose a new loss function that overcomes the shortcomings of original triplet ranking loss, thereby significantly improving retrieval performance. The proposed model has been evaluated on two public datasets and achieves competitive retrieval performance when compared with several state-of-the-art models. Implementation codes can be found here. |
| title | Enhancing Image-Text Matching with Adaptive Feature Aggregation |
| topic | Information Retrieval Multimedia |
| url | https://arxiv.org/abs/2401.09725 |