Recall: Empowering Multimodal Embedding for Edge Devices
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929512123465728 |
|---|---|
| author | Cai, Dongqi Wang, Shangguang Peng, Chen Zhang, Zeling Xu, Mengwei |
| author_facet | Cai, Dongqi Wang, Shangguang Peng, Chen Zhang, Zeling Xu, Mengwei |
| contents | Human memory is inherently prone to forgetting. To address this, multimodal embedding models have been introduced, which transform diverse real-world data into a unified embedding space. These embeddings can be retrieved efficiently, aiding mobile users in recalling past information. However, as model complexity grows, so do its resource demands, leading to reduced throughput and heavy computational requirements that limit mobile device implementation. In this paper, we introduce RECALL, a novel on-device multimodal embedding system optimized for resource-limited mobile environments. RECALL achieves high-throughput, accurate retrieval by generating coarse-grained embeddings and leveraging query-based filtering for refined retrieval. Experimental results demonstrate that RECALL delivers high-quality embeddings with superior throughput, all while operating unobtrusively with minimal memory and energy consumption. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_15342 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Recall: Empowering Multimodal Embedding for Edge Devices Cai, Dongqi Wang, Shangguang Peng, Chen Zhang, Zeling Xu, Mengwei Information Retrieval Artificial Intelligence Machine Learning Human memory is inherently prone to forgetting. To address this, multimodal embedding models have been introduced, which transform diverse real-world data into a unified embedding space. These embeddings can be retrieved efficiently, aiding mobile users in recalling past information. However, as model complexity grows, so do its resource demands, leading to reduced throughput and heavy computational requirements that limit mobile device implementation. In this paper, we introduce RECALL, a novel on-device multimodal embedding system optimized for resource-limited mobile environments. RECALL achieves high-throughput, accurate retrieval by generating coarse-grained embeddings and leveraging query-based filtering for refined retrieval. Experimental results demonstrate that RECALL delivers high-quality embeddings with superior throughput, all while operating unobtrusively with minimal memory and energy consumption. |
| title | Recall: Empowering Multimodal Embedding for Edge Devices |
| topic | Information Retrieval Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2409.15342 |