MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devices
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914625738506240 |
|---|---|
| author | Chu, Xiangxiang Qiao, Limeng Lin, Xinyang Xu, Shuang Yang, Yang Hu, Yiming Wei, Fei Zhang, Xinyu Zhang, Bo Wei, Xiaolin Shen, Chunhua |
| author_facet | Chu, Xiangxiang Qiao, Limeng Lin, Xinyang Xu, Shuang Yang, Yang Hu, Yiming Wei, Fei Zhang, Xinyu Zhang, Bo Wei, Xiaolin Shen, Chunhua |
| contents | We present MobileVLM, a competent multimodal vision language model (MMVLM) targeted to run on mobile devices. It is an amalgamation of a myriad of architectural designs and techniques that are mobile-oriented, which comprises a set of language models at the scale of 1.4B and 2.7B parameters, trained from scratch, a multimodal vision model that is pre-trained in the CLIP fashion, cross-modality interaction via an efficient projector. We evaluate MobileVLM on several typical VLM benchmarks. Our models demonstrate on par performance compared with a few much larger models. More importantly, we measure the inference speed on both a Qualcomm Snapdragon 888 CPU and an NVIDIA Jeston Orin GPU, and we obtain state-of-the-art performance of 21.5 tokens and 65.3 tokens per second, respectively. Our code will be made available at: https://github.com/Meituan-AutoML/MobileVLM. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_16886 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devices Chu, Xiangxiang Qiao, Limeng Lin, Xinyang Xu, Shuang Yang, Yang Hu, Yiming Wei, Fei Zhang, Xinyu Zhang, Bo Wei, Xiaolin Shen, Chunhua Computer Vision and Pattern Recognition We present MobileVLM, a competent multimodal vision language model (MMVLM) targeted to run on mobile devices. It is an amalgamation of a myriad of architectural designs and techniques that are mobile-oriented, which comprises a set of language models at the scale of 1.4B and 2.7B parameters, trained from scratch, a multimodal vision model that is pre-trained in the CLIP fashion, cross-modality interaction via an efficient projector. We evaluate MobileVLM on several typical VLM benchmarks. Our models demonstrate on par performance compared with a few much larger models. More importantly, we measure the inference speed on both a Qualcomm Snapdragon 888 CPU and an NVIDIA Jeston Orin GPU, and we obtain state-of-the-art performance of 21.5 tokens and 65.3 tokens per second, respectively. Our code will be made available at: https://github.com/Meituan-AutoML/MobileVLM. |
| title | MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devices |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2312.16886 |