Reason to Contrast: A Cascaded Multimodal Retrieval Framework
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914355494256640 |
|---|---|
| author | Cui, Xuanming Chen, Hong-You Yu, Hao Yuan, Hao Wang, Zihao Mishra, Shlok Kumar Yu, Hanchao Yang, Yonghuan Xiao, Jun Lim, Ser-Nam Cheng, Jianpeng Guo, Qi Fan, Xiangjun |
| author_facet | Cui, Xuanming Chen, Hong-You Yu, Hao Yuan, Hao Wang, Zihao Mishra, Shlok Kumar Yu, Hanchao Yang, Yonghuan Xiao, Jun Lim, Ser-Nam Cheng, Jianpeng Guo, Qi Fan, Xiangjun |
| contents | Traditional multimodal retrieval systems rely primarily on bi-encoder architectures, where performance is closely tied to embedding dimensionality. Recent work, Think-Then-Embed (TTE), shows that incorporating multimodal reasoning to elicit additional informative tokens before embedding can further improve retrieval. In this paper, we extend this paradigm with TTE-v2, a hybrid multimodal retrieval framework that introduces reasoning-driven performance scaling based on additional input token budget rather than model or embedding size. Our approach augments the initial multimodal retrieval with additional reasoning steps for reranking, enabling more expressive query-candidate interactions at test time. The reranking stage further provides fine-grained supervision for hard negative mining and false negative filtering, creating a feedback loop that effectively strengthens the upstream retriever. This cascaded design delivers substantial test-time improvements based on intermediate reasoning token scaling. Experiments on the MMEB-V2 benchmark demonstrate that TTE-v2-7B achieves a new state-of-the-art accuracy of 75.7%, and that TTE-v2-2B matches or surpasses leading 7B models trained with significantly larger external data. Our results highlight the promise of token-wise scaling as an alternative scaling paradigm for multimodal retrieval. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_23369 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Reason to Contrast: A Cascaded Multimodal Retrieval Framework Cui, Xuanming Chen, Hong-You Yu, Hao Yuan, Hao Wang, Zihao Mishra, Shlok Kumar Yu, Hanchao Yang, Yonghuan Xiao, Jun Lim, Ser-Nam Cheng, Jianpeng Guo, Qi Fan, Xiangjun Information Retrieval Artificial Intelligence Computation and Language Traditional multimodal retrieval systems rely primarily on bi-encoder architectures, where performance is closely tied to embedding dimensionality. Recent work, Think-Then-Embed (TTE), shows that incorporating multimodal reasoning to elicit additional informative tokens before embedding can further improve retrieval. In this paper, we extend this paradigm with TTE-v2, a hybrid multimodal retrieval framework that introduces reasoning-driven performance scaling based on additional input token budget rather than model or embedding size. Our approach augments the initial multimodal retrieval with additional reasoning steps for reranking, enabling more expressive query-candidate interactions at test time. The reranking stage further provides fine-grained supervision for hard negative mining and false negative filtering, creating a feedback loop that effectively strengthens the upstream retriever. This cascaded design delivers substantial test-time improvements based on intermediate reasoning token scaling. Experiments on the MMEB-V2 benchmark demonstrate that TTE-v2-7B achieves a new state-of-the-art accuracy of 75.7%, and that TTE-v2-2B matches or surpasses leading 7B models trained with significantly larger external data. Our results highlight the promise of token-wise scaling as an alternative scaling paradigm for multimodal retrieval. |
| title | Reason to Contrast: A Cascaded Multimodal Retrieval Framework |
| topic | Information Retrieval Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2602.23369 |