TTE-Flash: Accelerating Reasoning-based Multimodal Representations via Think-Then-Embed Tokens
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916018030379008 |
|---|---|
| author | Cheng, Jianpeng Wu, Xian Zhang, Jiangfan Bao, Wentao Ahuja, Chaitanya Mishra, Shlok Kumar Yu, Hanchao Gao, Yang Xia, Fan Guo, Qi Zhai, Shaodan Fan, Xiangjun Xiao, Jun |
| author_facet | Cheng, Jianpeng Wu, Xian Zhang, Jiangfan Bao, Wentao Ahuja, Chaitanya Mishra, Shlok Kumar Yu, Hanchao Gao, Yang Xia, Fan Guo, Qi Zhai, Shaodan Fan, Xiangjun Xiao, Jun |
| contents | Recent research has demonstrated that Universal Multimodal Embedding (UME) benefits significantly from Chain-of-Thought (CoT) reasoning. In this paradigm, a generative model produces explicit reasoning traces for a multimodal query, with the final representation extracted from an <eos> embedding token attending to both the query and the reasoning. Despite its effectiveness, the computational overhead of generating explicit CoT traces is often prohibitive. In this work, we propose replacing explicit CoT with latent think tokens, which are interpreted as latent variables that can produce explicit CoT traces as observed variables. By optimizing think tokens using CoT generation loss and subsequent embedding tokens using contrastive loss, we produce high-performance, reasoning-aware representations at a constant inference cost. Our study investigates two key architectural designs: 1) how think and embeddings tokens should be extracted from the same LLM backbone. 2) how the tokens should be trained as two dependent tasks. We introduce TTE-Flash-2B, a reasoning-aware multimodal representation model that outperforms its explicit-CoT counterpart on the MMEB-v2 benchmark, while producing latent think tokens that are interpretable both textually and visually. Furthermore, zero-shot evaluation across 15 video datasets reveals scaling behavior as the number of think tokens increases, and motivating a pilot study of adaptive think budget allocation based on task requirements. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_16638 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | TTE-Flash: Accelerating Reasoning-based Multimodal Representations via Think-Then-Embed Tokens Cheng, Jianpeng Wu, Xian Zhang, Jiangfan Bao, Wentao Ahuja, Chaitanya Mishra, Shlok Kumar Yu, Hanchao Gao, Yang Xia, Fan Guo, Qi Zhai, Shaodan Fan, Xiangjun Xiao, Jun Artificial Intelligence Recent research has demonstrated that Universal Multimodal Embedding (UME) benefits significantly from Chain-of-Thought (CoT) reasoning. In this paradigm, a generative model produces explicit reasoning traces for a multimodal query, with the final representation extracted from an <eos> embedding token attending to both the query and the reasoning. Despite its effectiveness, the computational overhead of generating explicit CoT traces is often prohibitive. In this work, we propose replacing explicit CoT with latent think tokens, which are interpreted as latent variables that can produce explicit CoT traces as observed variables. By optimizing think tokens using CoT generation loss and subsequent embedding tokens using contrastive loss, we produce high-performance, reasoning-aware representations at a constant inference cost. Our study investigates two key architectural designs: 1) how think and embeddings tokens should be extracted from the same LLM backbone. 2) how the tokens should be trained as two dependent tasks. We introduce TTE-Flash-2B, a reasoning-aware multimodal representation model that outperforms its explicit-CoT counterpart on the MMEB-v2 benchmark, while producing latent think tokens that are interpretable both textually and visually. Furthermore, zero-shot evaluation across 15 video datasets reveals scaling behavior as the number of think tokens increases, and motivating a pilot study of adaptive think budget allocation based on task requirements. |
| title | TTE-Flash: Accelerating Reasoning-based Multimodal Representations via Think-Then-Embed Tokens |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2605.16638 |