NaViL: Rethinking Scaling Properties of Native Multimodal Large Language Models under Data Constraints
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908585271754752 |
|---|---|
| author | Tian, Changyao Li, Hao Luo, Gen Zhu, Xizhou Su, Weijie Deng, Hanming Zhu, Jinguo Shao, Jie Zhu, Ziran Liu, Yunpeng Lu, Lewei Wang, Wenhai Li, Hongsheng Dai, Jifeng |
| author_facet | Tian, Changyao Li, Hao Luo, Gen Zhu, Xizhou Su, Weijie Deng, Hanming Zhu, Jinguo Shao, Jie Zhu, Ziran Liu, Yunpeng Lu, Lewei Wang, Wenhai Li, Hongsheng Dai, Jifeng |
| contents | Compositional training has been the de-facto paradigm in existing Multimodal Large Language Models (MLLMs), where pre-trained vision encoders are connected with pre-trained LLMs through continuous multimodal pre-training. However, the multimodal scaling property of this paradigm remains difficult to explore due to the separated training. In this paper, we focus on the native training of MLLMs in an end-to-end manner and systematically study its design space and scaling property under a practical setting, i.e., data constraint. Through careful study of various choices in MLLM, we obtain the optimal meta-architecture that best balances performance and training cost. After that, we further explore the scaling properties of the native MLLM and indicate the positively correlated scaling relationship between visual encoders and LLMs. Based on these findings, we propose a native MLLM called NaViL, combined with a simple and cost-effective recipe. Experimental results on 14 multimodal benchmarks confirm the competitive performance of NaViL against existing MLLMs. Besides that, our findings and results provide in-depth insights for the future study of native MLLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_08565 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | NaViL: Rethinking Scaling Properties of Native Multimodal Large Language Models under Data Constraints Tian, Changyao Li, Hao Luo, Gen Zhu, Xizhou Su, Weijie Deng, Hanming Zhu, Jinguo Shao, Jie Zhu, Ziran Liu, Yunpeng Lu, Lewei Wang, Wenhai Li, Hongsheng Dai, Jifeng Computer Vision and Pattern Recognition Compositional training has been the de-facto paradigm in existing Multimodal Large Language Models (MLLMs), where pre-trained vision encoders are connected with pre-trained LLMs through continuous multimodal pre-training. However, the multimodal scaling property of this paradigm remains difficult to explore due to the separated training. In this paper, we focus on the native training of MLLMs in an end-to-end manner and systematically study its design space and scaling property under a practical setting, i.e., data constraint. Through careful study of various choices in MLLM, we obtain the optimal meta-architecture that best balances performance and training cost. After that, we further explore the scaling properties of the native MLLM and indicate the positively correlated scaling relationship between visual encoders and LLMs. Based on these findings, we propose a native MLLM called NaViL, combined with a simple and cost-effective recipe. Experimental results on 14 multimodal benchmarks confirm the competitive performance of NaViL against existing MLLMs. Besides that, our findings and results provide in-depth insights for the future study of native MLLMs. |
| title | NaViL: Rethinking Scaling Properties of Native Multimodal Large Language Models under Data Constraints |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2510.08565 |