Test-time Vocabulary Adaptation for Language-driven Object Detection
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913868740034560 |
|---|---|
| author | Liu, Mingxuan Hayes, Tyler L. Mancini, Massimiliano Ricci, Elisa Volpi, Riccardo Csurka, Gabriela |
| author_facet | Liu, Mingxuan Hayes, Tyler L. Mancini, Massimiliano Ricci, Elisa Volpi, Riccardo Csurka, Gabriela |
| contents | Open-vocabulary object detection models allow users to freely specify a class vocabulary in natural language at test time, guiding the detection of desired objects. However, vocabularies can be overly broad or even mis-specified, hampering the overall performance of the detector. In this work, we propose a plug-and-play Vocabulary Adapter (VocAda) to refine the user-defined vocabulary, automatically tailoring it to categories that are relevant for a given image. VocAda does not require any training, it operates at inference time in three steps: i) it uses an image captionner to describe visible objects, ii) it parses nouns from those captions, and iii) it selects relevant classes from the user-defined vocabulary, discarding irrelevant ones. Experiments on COCO and Objects365 with three state-of-the-art detectors show that VocAda consistently improves performance, proving its versatility. The code is open source. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_00333 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Test-time Vocabulary Adaptation for Language-driven Object Detection Liu, Mingxuan Hayes, Tyler L. Mancini, Massimiliano Ricci, Elisa Volpi, Riccardo Csurka, Gabriela Computer Vision and Pattern Recognition Open-vocabulary object detection models allow users to freely specify a class vocabulary in natural language at test time, guiding the detection of desired objects. However, vocabularies can be overly broad or even mis-specified, hampering the overall performance of the detector. In this work, we propose a plug-and-play Vocabulary Adapter (VocAda) to refine the user-defined vocabulary, automatically tailoring it to categories that are relevant for a given image. VocAda does not require any training, it operates at inference time in three steps: i) it uses an image captionner to describe visible objects, ii) it parses nouns from those captions, and iii) it selects relevant classes from the user-defined vocabulary, discarding irrelevant ones. Experiments on COCO and Objects365 with three state-of-the-art detectors show that VocAda consistently improves performance, proving its versatility. The code is open source. |
| title | Test-time Vocabulary Adaptation for Language-driven Object Detection |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2506.00333 |