WRF4CIR: Weight-Regularized Fine-Tuning Network for Composed Image Retrieval
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908942195490816 |
|---|---|
| author | Xu, Yizhuo Yu, Chaojian Shao, Yuanjie Liu, Tongliang Peng, Qinmu You, Xinge |
| author_facet | Xu, Yizhuo Yu, Chaojian Shao, Yuanjie Liu, Tongliang Peng, Qinmu You, Xinge |
| contents | Composed Image Retrieval (CIR) task aims to retrieve target images based on reference images and modification texts. Current CIR methods primarily rely on fine-tuning vision-language pre-trained models. However, we find that these approaches commonly suffer from severe overfitting, posing challenges for CIR with limited triplet data. To better understand this issue, we present a systematic study of overfitting in VLP-based CIR, revealing a significant and previously overlooked generalization gap across different models and datasets. Motivated by these findings, we introduce WRF4CIR, a Weight-Regularized Fine-tuning network for CIR. Specifically, during the fine-tuning process, we apply adversarial perturbations to the model weights for regularization, where these perturbations are generated in the opposite direction of gradient descent. Intuitively, WRF4CIR increases the difficulty of fitting the training data, which helps mitigate overfitting in CIR under limited triplet supervision. Extensive experiments on benchmark datasets demonstrate that WRF4CIR significantly narrows the generalization gap and achieves substantial improvements over existing methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_05583 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | WRF4CIR: Weight-Regularized Fine-Tuning Network for Composed Image Retrieval Xu, Yizhuo Yu, Chaojian Shao, Yuanjie Liu, Tongliang Peng, Qinmu You, Xinge Computer Vision and Pattern Recognition Composed Image Retrieval (CIR) task aims to retrieve target images based on reference images and modification texts. Current CIR methods primarily rely on fine-tuning vision-language pre-trained models. However, we find that these approaches commonly suffer from severe overfitting, posing challenges for CIR with limited triplet data. To better understand this issue, we present a systematic study of overfitting in VLP-based CIR, revealing a significant and previously overlooked generalization gap across different models and datasets. Motivated by these findings, we introduce WRF4CIR, a Weight-Regularized Fine-tuning network for CIR. Specifically, during the fine-tuning process, we apply adversarial perturbations to the model weights for regularization, where these perturbations are generated in the opposite direction of gradient descent. Intuitively, WRF4CIR increases the difficulty of fitting the training data, which helps mitigate overfitting in CIR under limited triplet supervision. Extensive experiments on benchmark datasets demonstrate that WRF4CIR significantly narrows the generalization gap and achieves substantial improvements over existing methods. |
| title | WRF4CIR: Weight-Regularized Fine-Tuning Network for Composed Image Retrieval |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2604.05583 |