Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2505.15592 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908816955670528 |
|---|---|
| author | Avogaro, Niccolo Frick, Thomas Cinar, Yagmur G. Caraballo, Daniel Skura, Cezary Janicki, Filip M. Kluska, Piotr Ebouky, Brown Farronato, Nicola Scheidegger, Florian Malossi, Cristiano Schindler, Konrad Bartezzaghi, Andrea Assaf, Roy Rigotti, Mattia |
| author_facet | Avogaro, Niccolo Frick, Thomas Cinar, Yagmur G. Caraballo, Daniel Skura, Cezary Janicki, Filip M. Kluska, Piotr Ebouky, Brown Farronato, Nicola Scheidegger, Florian Malossi, Cristiano Schindler, Konrad Bartezzaghi, Andrea Assaf, Roy Rigotti, Mattia |
| contents | Large-scale pretrained vision backbones have transformed computer vision by providing powerful feature extractors that enable various downstream tasks, including training-free approaches like visual prompting for semantic segmentation. Despite their success in generic scenarios, these models often fall short when applied to specialized technical domains where the visual features differ significantly from their training distribution. To bridge this gap, we introduce VP Lab, a comprehensive iterative framework that enhances visual prompting for robust segmentation model development. At the core of VP Lab lies E-PEFT, a novel ensemble of parameter-efficient fine-tuning techniques specifically designed to adapt our visual prompting pipeline to specific domains in a manner that is both parameter- and data-efficient. Our approach not only surpasses the state-of-the-art in parameter-efficient fine-tuning for the Segment Anything Model (SAM), but also facilitates an interactive, near-real-time loop, allowing users to observe progressively improving results as they experiment within the framework. By integrating E-PEFT with visual prompting, we demonstrate a remarkable 50\% increase in semantic segmentation mIoU performance across various technical datasets using only 5 validated images, establishing a new paradigm for fast, efficient, and interactive model deployment in new, challenging domains. This work comes in the form of a demonstration. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_15592 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VP Lab: a PEFT-Enabled Visual Prompting Laboratory for Semantic Segmentation Avogaro, Niccolo Frick, Thomas Cinar, Yagmur G. Caraballo, Daniel Skura, Cezary Janicki, Filip M. Kluska, Piotr Ebouky, Brown Farronato, Nicola Scheidegger, Florian Malossi, Cristiano Schindler, Konrad Bartezzaghi, Andrea Assaf, Roy Rigotti, Mattia Computer Vision and Pattern Recognition Large-scale pretrained vision backbones have transformed computer vision by providing powerful feature extractors that enable various downstream tasks, including training-free approaches like visual prompting for semantic segmentation. Despite their success in generic scenarios, these models often fall short when applied to specialized technical domains where the visual features differ significantly from their training distribution. To bridge this gap, we introduce VP Lab, a comprehensive iterative framework that enhances visual prompting for robust segmentation model development. At the core of VP Lab lies E-PEFT, a novel ensemble of parameter-efficient fine-tuning techniques specifically designed to adapt our visual prompting pipeline to specific domains in a manner that is both parameter- and data-efficient. Our approach not only surpasses the state-of-the-art in parameter-efficient fine-tuning for the Segment Anything Model (SAM), but also facilitates an interactive, near-real-time loop, allowing users to observe progressively improving results as they experiment within the framework. By integrating E-PEFT with visual prompting, we demonstrate a remarkable 50\% increase in semantic segmentation mIoU performance across various technical datasets using only 5 validated images, establishing a new paradigm for fast, efficient, and interactive model deployment in new, challenging domains. This work comes in the form of a demonstration. |
| title | VP Lab: a PEFT-Enabled Visual Prompting Laboratory for Semantic Segmentation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.15592 |