Beyond Chain-of-Thought: Rewrite as a Universal Interface for Generative Multimodal Embeddings
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916057378193408 |
|---|---|
| author | Wu, Peixi Mei, Ke Ma, Feipeng Chai, Bosong Lan, Zhibin Zhao, Chenxi Yan, Shannan Chen, Jie Hu, Zhangchi Peng, Yansong Lin, Bo Zhou, Junjie Yin, Dacheng Wang, Tianyi Rao, Fengyun Lyu, Jing Li, Hebei Sun, Xiaoyan |
| author_facet | Wu, Peixi Mei, Ke Ma, Feipeng Chai, Bosong Lan, Zhibin Zhao, Chenxi Yan, Shannan Chen, Jie Hu, Zhangchi Peng, Yansong Lin, Bo Zhou, Junjie Yin, Dacheng Wang, Tianyi Rao, Fengyun Lyu, Jing Li, Hebei Sun, Xiaoyan |
| contents | Multimodal Large Language Models (MLLMs) have emerged as a promising foundation for universal multimodal embeddings. Recent studies have shown that reasoning-driven generative multimodal embeddings can outperform discriminative embeddings on several embedding tasks. However, Chain-of-Thought (CoT) reasoning tends to generate redundant thinking steps and introduce semantic ambiguity in the summarized answers in broader retrieval scenarios. To address this limitation, we propose Rewrite-driven Multimodal Embedding (RIME), a unified framework that jointly optimizes generation and embedding through a retrieval-friendly rewrite. Meanwhile, we present the Cross-Mode Alignment (CMA) to bridge the generative and discriminative embedding spaces, enabling flexible mutual retrieval to trade off efficiency and accuracy. Based on this, we also introduce Refine Reinforcement Learning (Refine-RL) that treats discriminative embeddings as stable semantic anchors to guide the rewrite optimization. Extensive experiments on MMEB-V2, MRMR and UVRB demonstrate that RIME substantially outperforms prior generative embedding models while significantly reducing the length of thinking. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_22280 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Beyond Chain-of-Thought: Rewrite as a Universal Interface for Generative Multimodal Embeddings Wu, Peixi Mei, Ke Ma, Feipeng Chai, Bosong Lan, Zhibin Zhao, Chenxi Yan, Shannan Chen, Jie Hu, Zhangchi Peng, Yansong Lin, Bo Zhou, Junjie Yin, Dacheng Wang, Tianyi Rao, Fengyun Lyu, Jing Li, Hebei Sun, Xiaoyan Computer Vision and Pattern Recognition Multimodal Large Language Models (MLLMs) have emerged as a promising foundation for universal multimodal embeddings. Recent studies have shown that reasoning-driven generative multimodal embeddings can outperform discriminative embeddings on several embedding tasks. However, Chain-of-Thought (CoT) reasoning tends to generate redundant thinking steps and introduce semantic ambiguity in the summarized answers in broader retrieval scenarios. To address this limitation, we propose Rewrite-driven Multimodal Embedding (RIME), a unified framework that jointly optimizes generation and embedding through a retrieval-friendly rewrite. Meanwhile, we present the Cross-Mode Alignment (CMA) to bridge the generative and discriminative embedding spaces, enabling flexible mutual retrieval to trade off efficiency and accuracy. Based on this, we also introduce Refine Reinforcement Learning (Refine-RL) that treats discriminative embeddings as stable semantic anchors to guide the rewrite optimization. Extensive experiments on MMEB-V2, MRMR and UVRB demonstrate that RIME substantially outperforms prior generative embedding models while significantly reducing the length of thinking. |
| title | Beyond Chain-of-Thought: Rewrite as a Universal Interface for Generative Multimodal Embeddings |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2604.22280 |