Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2504.10479 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912336973922304 |
|---|---|
| author | Zhu, Jinguo Wang, Weiyun Chen, Zhe Liu, Zhaoyang Ye, Shenglong Gu, Lixin Tian, Hao Duan, Yuchen Su, Weijie Shao, Jie Gao, Zhangwei Cui, Erfei Wang, Xuehui Cao, Yue Liu, Yangzhou Wei, Xingguang Zhang, Hongjie Wang, Haomin Xu, Weiye Li, Hao Wang, Jiahao Deng, Nianchen Li, Songze He, Yinan Jiang, Tan Luo, Jiapeng Wang, Yi He, Conghui Shi, Botian Zhang, Xingcheng Shao, Wenqi He, Junjun Xiong, Yingtong Qu, Wenwen Sun, Peng Jiao, Penglong Lv, Han Wu, Lijun Zhang, Kaipeng Deng, Huipeng Ge, Jiaye Chen, Kai Wang, Limin Dou, Min Lu, Lewei Zhu, Xizhou Lu, Tong Lin, Dahua Qiao, Yu Dai, Jifeng Wang, Wenhai |
| author_facet | Zhu, Jinguo Wang, Weiyun Chen, Zhe Liu, Zhaoyang Ye, Shenglong Gu, Lixin Tian, Hao Duan, Yuchen Su, Weijie Shao, Jie Gao, Zhangwei Cui, Erfei Wang, Xuehui Cao, Yue Liu, Yangzhou Wei, Xingguang Zhang, Hongjie Wang, Haomin Xu, Weiye Li, Hao Wang, Jiahao Deng, Nianchen Li, Songze He, Yinan Jiang, Tan Luo, Jiapeng Wang, Yi He, Conghui Shi, Botian Zhang, Xingcheng Shao, Wenqi He, Junjun Xiong, Yingtong Qu, Wenwen Sun, Peng Jiao, Penglong Lv, Han Wu, Lijun Zhang, Kaipeng Deng, Huipeng Ge, Jiaye Chen, Kai Wang, Limin Dou, Min Lu, Lewei Zhu, Xizhou Lu, Tong Lin, Dahua Qiao, Yu Dai, Jifeng Wang, Wenhai |
| contents | We introduce InternVL3, a significant advancement in the InternVL series featuring a native multimodal pre-training paradigm. Rather than adapting a text-only large language model (LLM) into a multimodal large language model (MLLM) that supports visual inputs, InternVL3 jointly acquires multimodal and linguistic capabilities from both diverse multimodal data and pure-text corpora during a single pre-training stage. This unified training paradigm effectively addresses the complexities and alignment challenges commonly encountered in conventional post-hoc training pipelines for MLLMs. To further improve performance and scalability, InternVL3 incorporates variable visual position encoding (V2PE) to support extended multimodal contexts, employs advanced post-training techniques such as supervised fine-tuning (SFT) and mixed preference optimization (MPO), and adopts test-time scaling strategies alongside an optimized training infrastructure. Extensive empirical evaluations demonstrate that InternVL3 delivers superior performance across a wide range of multi-modal tasks. In particular, InternVL3-78B achieves a score of 72.2 on the MMMU benchmark, setting a new state-of-the-art among open-source MLLMs. Its capabilities remain highly competitive with leading proprietary models, including ChatGPT-4o, Claude 3.5 Sonnet, and Gemini 2.5 Pro, while also maintaining strong pure-language proficiency. In pursuit of open-science principles, we will publicly release both the training data and model weights to foster further research and development in next-generation MLLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_10479 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models Zhu, Jinguo Wang, Weiyun Chen, Zhe Liu, Zhaoyang Ye, Shenglong Gu, Lixin Tian, Hao Duan, Yuchen Su, Weijie Shao, Jie Gao, Zhangwei Cui, Erfei Wang, Xuehui Cao, Yue Liu, Yangzhou Wei, Xingguang Zhang, Hongjie Wang, Haomin Xu, Weiye Li, Hao Wang, Jiahao Deng, Nianchen Li, Songze He, Yinan Jiang, Tan Luo, Jiapeng Wang, Yi He, Conghui Shi, Botian Zhang, Xingcheng Shao, Wenqi He, Junjun Xiong, Yingtong Qu, Wenwen Sun, Peng Jiao, Penglong Lv, Han Wu, Lijun Zhang, Kaipeng Deng, Huipeng Ge, Jiaye Chen, Kai Wang, Limin Dou, Min Lu, Lewei Zhu, Xizhou Lu, Tong Lin, Dahua Qiao, Yu Dai, Jifeng Wang, Wenhai Computer Vision and Pattern Recognition We introduce InternVL3, a significant advancement in the InternVL series featuring a native multimodal pre-training paradigm. Rather than adapting a text-only large language model (LLM) into a multimodal large language model (MLLM) that supports visual inputs, InternVL3 jointly acquires multimodal and linguistic capabilities from both diverse multimodal data and pure-text corpora during a single pre-training stage. This unified training paradigm effectively addresses the complexities and alignment challenges commonly encountered in conventional post-hoc training pipelines for MLLMs. To further improve performance and scalability, InternVL3 incorporates variable visual position encoding (V2PE) to support extended multimodal contexts, employs advanced post-training techniques such as supervised fine-tuning (SFT) and mixed preference optimization (MPO), and adopts test-time scaling strategies alongside an optimized training infrastructure. Extensive empirical evaluations demonstrate that InternVL3 delivers superior performance across a wide range of multi-modal tasks. In particular, InternVL3-78B achieves a score of 72.2 on the MMMU benchmark, setting a new state-of-the-art among open-source MLLMs. Its capabilities remain highly competitive with leading proprietary models, including ChatGPT-4o, Claude 3.5 Sonnet, and Gemini 2.5 Pro, while also maintaining strong pure-language proficiency. In pursuit of open-science principles, we will publicly release both the training data and model weights to foster further research and development in next-generation MLLMs. |
| title | InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.10479 |