What Holds Back Open-Vocabulary Segmentation?
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908479895109632 |
|---|---|
| author | Šarić, Josip Martinović, Ivan Kristan, Matej Šegvić, Siniša |
| author_facet | Šarić, Josip Martinović, Ivan Kristan, Matej Šegvić, Siniša |
| contents | Standard segmentation setups are unable to deliver models that can recognize concepts outside the training taxonomy. Open-vocabulary approaches promise to close this gap through language-image pretraining on billions of image-caption pairs. Unfortunately, we observe that the promise is not delivered due to several bottlenecks that have caused the performance to plateau for almost two years. This paper proposes novel oracle components that identify and decouple these bottlenecks by taking advantage of the groundtruth information. The presented validation experiments deliver important empirical findings that provide a deeper insight into the failures of open-vocabulary models and suggest prominent approaches to unlock the future research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_04211 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | What Holds Back Open-Vocabulary Segmentation? Šarić, Josip Martinović, Ivan Kristan, Matej Šegvić, Siniša Computer Vision and Pattern Recognition Standard segmentation setups are unable to deliver models that can recognize concepts outside the training taxonomy. Open-vocabulary approaches promise to close this gap through language-image pretraining on billions of image-caption pairs. Unfortunately, we observe that the promise is not delivered due to several bottlenecks that have caused the performance to plateau for almost two years. This paper proposes novel oracle components that identify and decouple these bottlenecks by taking advantage of the groundtruth information. The presented validation experiments deliver important empirical findings that provide a deeper insight into the failures of open-vocabulary models and suggest prominent approaches to unlock the future research. |
| title | What Holds Back Open-Vocabulary Segmentation? |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2508.04211 |