VISTA: A Visual Analytics Framework to Enhance Foundation Model-Generated Data Labels
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909686299623424 |
|---|---|
| author | Xuan, Xiwei Wang, Xiaoqi He, Wenbin Ono, Jorge Piazentin Gou, Liang Ma, Kwan-Liu Ren, Liu |
| author_facet | Xuan, Xiwei Wang, Xiaoqi He, Wenbin Ono, Jorge Piazentin Gou, Liang Ma, Kwan-Liu Ren, Liu |
| contents | The advances in multi-modal foundation models (FMs) (e.g., CLIP and LLaVA) have facilitated the auto-labeling of large-scale datasets, enhancing model performance in challenging downstream tasks such as open-vocabulary object detection and segmentation. However, the quality of FM-generated labels is less studied as existing approaches focus more on data quantity over quality. This is because validating large volumes of data without ground truth presents a considerable challenge in practice. Existing methods typically rely on limited metrics to identify problematic data, lacking a comprehensive perspective, or apply human validation to only a small data fraction, failing to address the full spectrum of potential issues. To overcome these challenges, we introduce VISTA, a visual analytics framework that improves data quality to enhance the performance of multi-modal models. Targeting the complex and demanding domain of open-vocabulary image segmentation, VISTA integrates multi-phased data validation strategies with human expertise, enabling humans to identify, understand, and correct hidden issues within FM-generated labels. Through detailed use cases on two benchmark datasets and expert reviews, we demonstrate VISTA's effectiveness from both quantitative and qualitative perspectives. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_09008 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VISTA: A Visual Analytics Framework to Enhance Foundation Model-Generated Data Labels Xuan, Xiwei Wang, Xiaoqi He, Wenbin Ono, Jorge Piazentin Gou, Liang Ma, Kwan-Liu Ren, Liu Computer Vision and Pattern Recognition The advances in multi-modal foundation models (FMs) (e.g., CLIP and LLaVA) have facilitated the auto-labeling of large-scale datasets, enhancing model performance in challenging downstream tasks such as open-vocabulary object detection and segmentation. However, the quality of FM-generated labels is less studied as existing approaches focus more on data quantity over quality. This is because validating large volumes of data without ground truth presents a considerable challenge in practice. Existing methods typically rely on limited metrics to identify problematic data, lacking a comprehensive perspective, or apply human validation to only a small data fraction, failing to address the full spectrum of potential issues. To overcome these challenges, we introduce VISTA, a visual analytics framework that improves data quality to enhance the performance of multi-modal models. Targeting the complex and demanding domain of open-vocabulary image segmentation, VISTA integrates multi-phased data validation strategies with human expertise, enabling humans to identify, understand, and correct hidden issues within FM-generated labels. Through detailed use cases on two benchmark datasets and expert reviews, we demonstrate VISTA's effectiveness from both quantitative and qualitative perspectives. |
| title | VISTA: A Visual Analytics Framework to Enhance Foundation Model-Generated Data Labels |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2507.09008 |