InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916093931552768 |
|---|---|
| author | Chen, Zhe Wu, Jiannan Wang, Wenhai Su, Weijie Chen, Guo Xing, Sen Zhong, Muyan Zhang, Qinglong Zhu, Xizhou Lu, Lewei Li, Bin Luo, Ping Lu, Tong Qiao, Yu Dai, Jifeng |
| author_facet | Chen, Zhe Wu, Jiannan Wang, Wenhai Su, Weijie Chen, Guo Xing, Sen Zhong, Muyan Zhang, Qinglong Zhu, Xizhou Lu, Lewei Li, Bin Luo, Ping Lu, Tong Qiao, Yu Dai, Jifeng |
| contents | The exponential growth of large language models (LLMs) has opened up numerous possibilities for multimodal AGI systems. However, the progress in vision and vision-language foundation models, which are also critical elements of multi-modal AGI, has not kept pace with LLMs. In this work, we design a large-scale vision-language foundation model (InternVL), which scales up the vision foundation model to 6 billion parameters and progressively aligns it with the LLM, using web-scale image-text data from various sources. This model can be broadly applied to and achieve state-of-the-art performance on 32 generic visual-linguistic benchmarks including visual perception tasks such as image-level or pixel-level recognition, vision-language tasks such as zero-shot image/video classification, zero-shot image/video-text retrieval, and link with LLMs to create multi-modal dialogue systems. It has powerful visual capabilities and can be a good alternative to the ViT-22B. We hope that our research could contribute to the development of multi-modal large models. Code and models are available at https://github.com/OpenGVLab/InternVL. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_14238 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks Chen, Zhe Wu, Jiannan Wang, Wenhai Su, Weijie Chen, Guo Xing, Sen Zhong, Muyan Zhang, Qinglong Zhu, Xizhou Lu, Lewei Li, Bin Luo, Ping Lu, Tong Qiao, Yu Dai, Jifeng Computer Vision and Pattern Recognition The exponential growth of large language models (LLMs) has opened up numerous possibilities for multimodal AGI systems. However, the progress in vision and vision-language foundation models, which are also critical elements of multi-modal AGI, has not kept pace with LLMs. In this work, we design a large-scale vision-language foundation model (InternVL), which scales up the vision foundation model to 6 billion parameters and progressively aligns it with the LLM, using web-scale image-text data from various sources. This model can be broadly applied to and achieve state-of-the-art performance on 32 generic visual-linguistic benchmarks including visual perception tasks such as image-level or pixel-level recognition, vision-language tasks such as zero-shot image/video classification, zero-shot image/video-text retrieval, and link with LLMs to create multi-modal dialogue systems. It has powerful visual capabilities and can be a good alternative to the ViT-22B. We hope that our research could contribute to the development of multi-modal large models. Code and models are available at https://github.com/OpenGVLab/InternVL. |
| title | InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2312.14238 |