Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.05271 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909807441608704 |
|---|---|
| author | Chen, Zhe Wang, Weiyun Cao, Yue Liu, Yangzhou Gao, Zhangwei Cui, Erfei Zhu, Jinguo Ye, Shenglong Tian, Hao Liu, Zhaoyang Gu, Lixin Wang, Xuehui Li, Qingyun Ren, Yiming Chen, Zixuan Luo, Jiapeng Wang, Jiahao Jiang, Tan Wang, Bo He, Conghui Shi, Botian Zhang, Xingcheng Lv, Han Wang, Yi Shao, Wenqi Chu, Pei Tu, Zhongying He, Tong Wu, Zhiyong Deng, Huipeng Ge, Jiaye Chen, Kai Zhang, Kaipeng Wang, Limin Dou, Min Lu, Lewei Zhu, Xizhou Lu, Tong Lin, Dahua Qiao, Yu Dai, Jifeng Wang, Wenhai |
| author_facet | Chen, Zhe Wang, Weiyun Cao, Yue Liu, Yangzhou Gao, Zhangwei Cui, Erfei Zhu, Jinguo Ye, Shenglong Tian, Hao Liu, Zhaoyang Gu, Lixin Wang, Xuehui Li, Qingyun Ren, Yiming Chen, Zixuan Luo, Jiapeng Wang, Jiahao Jiang, Tan Wang, Bo He, Conghui Shi, Botian Zhang, Xingcheng Lv, Han Wang, Yi Shao, Wenqi Chu, Pei Tu, Zhongying He, Tong Wu, Zhiyong Deng, Huipeng Ge, Jiaye Chen, Kai Zhang, Kaipeng Wang, Limin Dou, Min Lu, Lewei Zhu, Xizhou Lu, Tong Lin, Dahua Qiao, Yu Dai, Jifeng Wang, Wenhai |
| contents | We introduce InternVL 2.5, an advanced multimodal large language model (MLLM) series that builds upon InternVL 2.0, maintaining its core model architecture while introducing significant enhancements in training and testing strategies as well as data quality. In this work, we delve into the relationship between model scaling and performance, systematically exploring the performance trends in vision encoders, language models, dataset sizes, and test-time configurations. Through extensive evaluations on a wide range of benchmarks, including multi-discipline reasoning, document understanding, multi-image / video understanding, real-world comprehension, multimodal hallucination detection, visual grounding, multilingual capabilities, and pure language processing, InternVL 2.5 exhibits competitive performance, rivaling leading commercial models such as GPT-4o and Claude-3.5-Sonnet. Notably, our model is the first open-source MLLMs to surpass 70% on the MMMU benchmark, achieving a 3.7-point improvement through Chain-of-Thought (CoT) reasoning and showcasing strong potential for test-time scaling. We hope this model contributes to the open-source community by setting new standards for developing and applying multimodal AI systems. HuggingFace demo see https://huggingface.co/spaces/OpenGVLab/InternVL |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_05271 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling Chen, Zhe Wang, Weiyun Cao, Yue Liu, Yangzhou Gao, Zhangwei Cui, Erfei Zhu, Jinguo Ye, Shenglong Tian, Hao Liu, Zhaoyang Gu, Lixin Wang, Xuehui Li, Qingyun Ren, Yiming Chen, Zixuan Luo, Jiapeng Wang, Jiahao Jiang, Tan Wang, Bo He, Conghui Shi, Botian Zhang, Xingcheng Lv, Han Wang, Yi Shao, Wenqi Chu, Pei Tu, Zhongying He, Tong Wu, Zhiyong Deng, Huipeng Ge, Jiaye Chen, Kai Zhang, Kaipeng Wang, Limin Dou, Min Lu, Lewei Zhu, Xizhou Lu, Tong Lin, Dahua Qiao, Yu Dai, Jifeng Wang, Wenhai Computer Vision and Pattern Recognition We introduce InternVL 2.5, an advanced multimodal large language model (MLLM) series that builds upon InternVL 2.0, maintaining its core model architecture while introducing significant enhancements in training and testing strategies as well as data quality. In this work, we delve into the relationship between model scaling and performance, systematically exploring the performance trends in vision encoders, language models, dataset sizes, and test-time configurations. Through extensive evaluations on a wide range of benchmarks, including multi-discipline reasoning, document understanding, multi-image / video understanding, real-world comprehension, multimodal hallucination detection, visual grounding, multilingual capabilities, and pure language processing, InternVL 2.5 exhibits competitive performance, rivaling leading commercial models such as GPT-4o and Claude-3.5-Sonnet. Notably, our model is the first open-source MLLMs to surpass 70% on the MMMU benchmark, achieving a 3.7-point improvement through Chain-of-Thought (CoT) reasoning and showcasing strong potential for test-time scaling. We hope this model contributes to the open-source community by setting new standards for developing and applying multimodal AI systems. HuggingFace demo see https://huggingface.co/spaces/OpenGVLab/InternVL |
| title | Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.05271 |