RESMatch: Referring Expression Segmentation in a Semi-Supervised Manner
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914673452908544 |
|---|---|
| author | Zang, Ying Fu, Chenglong Cao, Runlong Zhu, Didi Zhang, Min Hu, Wenjun Zhu, Lanyun Chen, Tianrun |
| author_facet | Zang, Ying Fu, Chenglong Cao, Runlong Zhu, Didi Zhang, Min Hu, Wenjun Zhu, Lanyun Chen, Tianrun |
| contents | Referring expression segmentation (RES), a task that involves localizing specific instance-level objects based on free-form linguistic descriptions, has emerged as a crucial frontier in human-AI interaction. It demands an intricate understanding of both visual and textual contexts and often requires extensive training data. This paper introduces RESMatch, the first semi-supervised learning (SSL) approach for RES, aimed at reducing reliance on exhaustive data annotation. Extensive validation on multiple RES datasets demonstrates that RESMatch significantly outperforms baseline approaches, establishing a new state-of-the-art. Although existing SSL techniques are effective in image segmentation, we find that they fall short in RES. Facing the challenges including the comprehension of free-form linguistic descriptions and the variability in object attributes, RESMatch introduces a trifecta of adaptations: revised strong perturbation, text augmentation, and adjustments for pseudo-label quality and strong-weak supervision. This pioneering work lays the groundwork for future research in semi-supervised learning for referring expression segmentation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_05589 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | RESMatch: Referring Expression Segmentation in a Semi-Supervised Manner Zang, Ying Fu, Chenglong Cao, Runlong Zhu, Didi Zhang, Min Hu, Wenjun Zhu, Lanyun Chen, Tianrun Computer Vision and Pattern Recognition Referring expression segmentation (RES), a task that involves localizing specific instance-level objects based on free-form linguistic descriptions, has emerged as a crucial frontier in human-AI interaction. It demands an intricate understanding of both visual and textual contexts and often requires extensive training data. This paper introduces RESMatch, the first semi-supervised learning (SSL) approach for RES, aimed at reducing reliance on exhaustive data annotation. Extensive validation on multiple RES datasets demonstrates that RESMatch significantly outperforms baseline approaches, establishing a new state-of-the-art. Although existing SSL techniques are effective in image segmentation, we find that they fall short in RES. Facing the challenges including the comprehension of free-form linguistic descriptions and the variability in object attributes, RESMatch introduces a trifecta of adaptations: revised strong perturbation, text augmentation, and adjustments for pseudo-label quality and strong-weak supervision. This pioneering work lays the groundwork for future research in semi-supervised learning for referring expression segmentation. |
| title | RESMatch: Referring Expression Segmentation in a Semi-Supervised Manner |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2402.05589 |