Aria: An Open Multimodal Native Mixture-of-Experts Model
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912182577397760 |
|---|---|
| author | Li, Dongxu Liu, Yudong Wu, Haoning Wang, Yue Shen, Zhiqi Qu, Bowen Niu, Xinyao Zhou, Fan Huang, Chengen Li, Yanpeng Zhu, Chongyan Ren, Xiaoyi Li, Chao Ye, Yifan Liu, Peng Zhang, Lihuan Yan, Hanshu Wang, Guoyin Chen, Bei Li, Junnan |
| author_facet | Li, Dongxu Liu, Yudong Wu, Haoning Wang, Yue Shen, Zhiqi Qu, Bowen Niu, Xinyao Zhou, Fan Huang, Chengen Li, Yanpeng Zhu, Chongyan Ren, Xiaoyi Li, Chao Ye, Yifan Liu, Peng Zhang, Lihuan Yan, Hanshu Wang, Guoyin Chen, Bei Li, Junnan |
| contents | Information comes in diverse modalities. Multimodal native AI models are essential to integrate real-world information and deliver comprehensive understanding. While proprietary multimodal native models exist, their lack of openness imposes obstacles for adoptions, let alone adaptations. To fill this gap, we introduce Aria, an open multimodal native model with best-in-class performance across a wide range of multimodal, language, and coding tasks. Aria is a mixture-of-expert model with 3.9B and 3.5B activated parameters per visual token and text token, respectively. It outperforms Pixtral-12B and Llama3.2-11B, and is competitive against the best proprietary models on various multimodal tasks. We pre-train Aria from scratch following a 4-stage pipeline, which progressively equips the model with strong capabilities in language understanding, multimodal understanding, long context window, and instruction following. We open-source the model weights along with a codebase that facilitates easy adoptions and adaptations of Aria in real-world applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_05993 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Aria: An Open Multimodal Native Mixture-of-Experts Model Li, Dongxu Liu, Yudong Wu, Haoning Wang, Yue Shen, Zhiqi Qu, Bowen Niu, Xinyao Zhou, Fan Huang, Chengen Li, Yanpeng Zhu, Chongyan Ren, Xiaoyi Li, Chao Ye, Yifan Liu, Peng Zhang, Lihuan Yan, Hanshu Wang, Guoyin Chen, Bei Li, Junnan Computer Vision and Pattern Recognition Information comes in diverse modalities. Multimodal native AI models are essential to integrate real-world information and deliver comprehensive understanding. While proprietary multimodal native models exist, their lack of openness imposes obstacles for adoptions, let alone adaptations. To fill this gap, we introduce Aria, an open multimodal native model with best-in-class performance across a wide range of multimodal, language, and coding tasks. Aria is a mixture-of-expert model with 3.9B and 3.5B activated parameters per visual token and text token, respectively. It outperforms Pixtral-12B and Llama3.2-11B, and is competitive against the best proprietary models on various multimodal tasks. We pre-train Aria from scratch following a 4-stage pipeline, which progressively equips the model with strong capabilities in language understanding, multimodal understanding, long context window, and instruction following. We open-source the model weights along with a codebase that facilitates easy adoptions and adaptations of Aria in real-world applications. |
| title | Aria: An Open Multimodal Native Mixture-of-Experts Model |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2410.05993 |