Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2402.11297 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909111016226816 |
|---|---|
| author | Zolkepli, Husein Razak, Aisyah Adha, Kamarul Nazhan, Ariff |
| author_facet | Zolkepli, Husein Razak, Aisyah Adha, Kamarul Nazhan, Ariff |
| contents | Our contribution introduces a groundbreaking multimodal large language model designed to comprehend multi-images, multi-audio, and multi-images-multi-audio within a single multiturn session. Leveraging state-of-the-art models, we utilize the SigLIP encoder for visual inputs and the Whisper Encoder for audio inputs. Notably, this multimodal large language model is bilingual, proficient in understanding both English and Malay simultaneously. We proudly unveil two versions of this model: TinyLlama with 1.1B parameters, and Mistral with 7B parameters. With its ability to navigate diverse modalities and languages, our model represents a significant advancement for the Malaysian context and beyond.
All models released at https://huggingface.co/collections/mesolitica/multimodal-malaysian-llm-65c6f893e03f78fa9e5c8859 |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_11297 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MMMModal -- Multi-Images Multi-Audio Multi-turn Multi-Modal Zolkepli, Husein Razak, Aisyah Adha, Kamarul Nazhan, Ariff Computation and Language Our contribution introduces a groundbreaking multimodal large language model designed to comprehend multi-images, multi-audio, and multi-images-multi-audio within a single multiturn session. Leveraging state-of-the-art models, we utilize the SigLIP encoder for visual inputs and the Whisper Encoder for audio inputs. Notably, this multimodal large language model is bilingual, proficient in understanding both English and Malay simultaneously. We proudly unveil two versions of this model: TinyLlama with 1.1B parameters, and Mistral with 7B parameters. With its ability to navigate diverse modalities and languages, our model represents a significant advancement for the Malaysian context and beyond. All models released at https://huggingface.co/collections/mesolitica/multimodal-malaysian-llm-65c6f893e03f78fa9e5c8859 |
| title | MMMModal -- Multi-Images Multi-Audio Multi-turn Multi-Modal |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2402.11297 |