Mitigating Noisy Correspondence by Geometrical Structure Consistency Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929359901687808 |
|---|---|
| author | Zhao, Zihua Chen, Mengxi Dai, Tianjie Yao, Jiangchao han, Bo Zhang, Ya Wang, Yanfeng |
| author_facet | Zhao, Zihua Chen, Mengxi Dai, Tianjie Yao, Jiangchao han, Bo Zhang, Ya Wang, Yanfeng |
| contents | Noisy correspondence that refers to mismatches in cross-modal data pairs, is prevalent on human-annotated or web-crawled datasets. Prior approaches to leverage such data mainly consider the application of uni-modal noisy label learning without amending the impact on both cross-modal and intra-modal geometrical structures in multimodal learning. Actually, we find that both structures are effective to discriminate noisy correspondence through structural differences when being well-established. Inspired by this observation, we introduce a Geometrical Structure Consistency (GSC) method to infer the true correspondence. Specifically, GSC ensures the preservation of geometrical structures within and between modalities, allowing for the accurate discrimination of noisy samples based on structural differences. Utilizing these inferred true correspondence labels, GSC refines the learning of geometrical structures by filtering out the noisy samples. Experiments across four cross-modal datasets confirm that GSC effectively identifies noisy samples and significantly outperforms the current leading methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_16996 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Mitigating Noisy Correspondence by Geometrical Structure Consistency Learning Zhao, Zihua Chen, Mengxi Dai, Tianjie Yao, Jiangchao han, Bo Zhang, Ya Wang, Yanfeng Computer Vision and Pattern Recognition Noisy correspondence that refers to mismatches in cross-modal data pairs, is prevalent on human-annotated or web-crawled datasets. Prior approaches to leverage such data mainly consider the application of uni-modal noisy label learning without amending the impact on both cross-modal and intra-modal geometrical structures in multimodal learning. Actually, we find that both structures are effective to discriminate noisy correspondence through structural differences when being well-established. Inspired by this observation, we introduce a Geometrical Structure Consistency (GSC) method to infer the true correspondence. Specifically, GSC ensures the preservation of geometrical structures within and between modalities, allowing for the accurate discrimination of noisy samples based on structural differences. Utilizing these inferred true correspondence labels, GSC refines the learning of geometrical structures by filtering out the noisy samples. Experiments across four cross-modal datasets confirm that GSC effectively identifies noisy samples and significantly outperforms the current leading methods. |
| title | Mitigating Noisy Correspondence by Geometrical Structure Consistency Learning |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2405.16996 |