Embracing Collaboration Over Competition: Condensing Multiple Prompts for Visual In-Context Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916714464149504 |
|---|---|
| author | Wang, Jinpeng Luo, Tianci Zha, Yaohua Feng, Yan Luo, Ruisheng Chen, Bin Dai, Tao Chen, Long Wang, Yaowei Xia, Shu-Tao |
| author_facet | Wang, Jinpeng Luo, Tianci Zha, Yaohua Feng, Yan Luo, Ruisheng Chen, Bin Dai, Tao Chen, Long Wang, Yaowei Xia, Shu-Tao |
| contents | Visual In-Context Learning (VICL) enables adaptively solving vision tasks by leveraging pixel demonstrations, mimicking human-like task completion through analogy. Prompt selection is critical in VICL, but current methods assume the existence of a single "ideal" prompt in a pool of candidates, which in practice may not hold true. Multiple suitable prompts may exist, but individually they often fall short, leading to difficulties in selection and the exclusion of useful context. To address this, we propose a new perspective: prompt condensation. Rather than relying on a single prompt, candidate prompts collaborate to efficiently integrate informative contexts without sacrificing resolution. We devise Condenser, a lightweight external plugin that compresses relevant fine-grained context across multiple prompts. Optimized end-to-end with the backbone, Condenser ensures accurate integration of contextual cues. Experiments demonstrate Condenser outperforms state-of-the-arts across benchmark tasks, showing superior context compression, scalability with more prompts, and enhanced computational efficiency compared to ensemble methods, positioning it as a highly competitive solution for VICL. Code is open-sourced at https://github.com/gimpong/CVPR25-Condenser. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_21263 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Embracing Collaboration Over Competition: Condensing Multiple Prompts for Visual In-Context Learning Wang, Jinpeng Luo, Tianci Zha, Yaohua Feng, Yan Luo, Ruisheng Chen, Bin Dai, Tao Chen, Long Wang, Yaowei Xia, Shu-Tao Computer Vision and Pattern Recognition Machine Learning Multimedia Visual In-Context Learning (VICL) enables adaptively solving vision tasks by leveraging pixel demonstrations, mimicking human-like task completion through analogy. Prompt selection is critical in VICL, but current methods assume the existence of a single "ideal" prompt in a pool of candidates, which in practice may not hold true. Multiple suitable prompts may exist, but individually they often fall short, leading to difficulties in selection and the exclusion of useful context. To address this, we propose a new perspective: prompt condensation. Rather than relying on a single prompt, candidate prompts collaborate to efficiently integrate informative contexts without sacrificing resolution. We devise Condenser, a lightweight external plugin that compresses relevant fine-grained context across multiple prompts. Optimized end-to-end with the backbone, Condenser ensures accurate integration of contextual cues. Experiments demonstrate Condenser outperforms state-of-the-arts across benchmark tasks, showing superior context compression, scalability with more prompts, and enhanced computational efficiency compared to ensemble methods, positioning it as a highly competitive solution for VICL. Code is open-sourced at https://github.com/gimpong/CVPR25-Condenser. |
| title | Embracing Collaboration Over Competition: Condensing Multiple Prompts for Visual In-Context Learning |
| topic | Computer Vision and Pattern Recognition Machine Learning Multimedia |
| url | https://arxiv.org/abs/2504.21263 |