Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.12760 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911547859664896 |
|---|---|
| author | Li, Xiaoyu Liu, Yuhang Kang, Xuanshuo Luo, Zheng Lou, Fangqi Wu, Xiaohua Xiong, Zihan |
| author_facet | Li, Xiaoyu Liu, Yuhang Kang, Xuanshuo Luo, Zheng Lou, Fangqi Wu, Xiaohua Xiong, Zihan |
| contents | In-Context Learning (ICL) is a significant paradigm for Large Multimodal Models (LMMs), using a few in-context demonstrations (ICDs) for new task adaptation. However, its performance is sensitive to demonstration configurations and computationally expensive. Mathematically, the influence of these demonstrations can be decomposed into a dynamic mixture of the standard attention output and the context values. Current approximation methods simplify this process by learning a "shift vector". Inspired by the exact decomposition, we introduce High-Fidelity In-Context Learning (HIFICL) to more faithfully model the ICL mechanism. HIFICL consists of three key components: 1) a set of "virtual key-value pairs" to act as a learnable context, 2) a low-rank factorization for stable and regularized training, and 3) a simple end-to-end training objective. From another perspective, this mechanism constitutes a form of context-aware Parameter-Efficient Fine-Tuning (PEFT). Extensive experiments show that HiFICL consistently outperforms existing approximation methods on several multimodal benchmarks. The code is available at https://github.com/bbbandari/HiFICL. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_12760 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | HIFICL: High-Fidelity In-Context Learning for Multimodal Tasks Li, Xiaoyu Liu, Yuhang Kang, Xuanshuo Luo, Zheng Lou, Fangqi Wu, Xiaohua Xiong, Zihan Computer Vision and Pattern Recognition In-Context Learning (ICL) is a significant paradigm for Large Multimodal Models (LMMs), using a few in-context demonstrations (ICDs) for new task adaptation. However, its performance is sensitive to demonstration configurations and computationally expensive. Mathematically, the influence of these demonstrations can be decomposed into a dynamic mixture of the standard attention output and the context values. Current approximation methods simplify this process by learning a "shift vector". Inspired by the exact decomposition, we introduce High-Fidelity In-Context Learning (HIFICL) to more faithfully model the ICL mechanism. HIFICL consists of three key components: 1) a set of "virtual key-value pairs" to act as a learnable context, 2) a low-rank factorization for stable and regularized training, and 3) a simple end-to-end training objective. From another perspective, this mechanism constitutes a form of context-aware Parameter-Efficient Fine-Tuning (PEFT). Extensive experiments show that HiFICL consistently outperforms existing approximation methods on several multimodal benchmarks. The code is available at https://github.com/bbbandari/HiFICL. |
| title | HIFICL: High-Fidelity In-Context Learning for Multimodal Tasks |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.12760 |