LLaVA-OneVision: Easy Visual Task Transfer
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909366015229952 |
|---|---|
| author | Li, Bo Zhang, Yuanhan Guo, Dong Zhang, Renrui Li, Feng Zhang, Hao Zhang, Kaichen Zhang, Peiyuan Li, Yanwei Liu, Ziwei Li, Chunyuan |
| author_facet | Li, Bo Zhang, Yuanhan Guo, Dong Zhang, Renrui Li, Feng Zhang, Hao Zhang, Kaichen Zhang, Peiyuan Li, Yanwei Liu, Ziwei Li, Chunyuan |
| contents | We present LLaVA-OneVision, a family of open large multimodal models (LMMs) developed by consolidating our insights into data, models, and visual representations in the LLaVA-NeXT blog series. Our experimental results demonstrate that LLaVA-OneVision is the first single model that can simultaneously push the performance boundaries of open LMMs in three important computer vision scenarios: single-image, multi-image, and video scenarios. Importantly, the design of LLaVA-OneVision allows strong transfer learning across different modalities/scenarios, yielding new emerging capabilities. In particular, strong video understanding and cross-scenario capabilities are demonstrated through task transfer from images to videos. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_03326 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LLaVA-OneVision: Easy Visual Task Transfer Li, Bo Zhang, Yuanhan Guo, Dong Zhang, Renrui Li, Feng Zhang, Hao Zhang, Kaichen Zhang, Peiyuan Li, Yanwei Liu, Ziwei Li, Chunyuan Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language We present LLaVA-OneVision, a family of open large multimodal models (LMMs) developed by consolidating our insights into data, models, and visual representations in the LLaVA-NeXT blog series. Our experimental results demonstrate that LLaVA-OneVision is the first single model that can simultaneously push the performance boundaries of open LMMs in three important computer vision scenarios: single-image, multi-image, and video scenarios. Importantly, the design of LLaVA-OneVision allows strong transfer learning across different modalities/scenarios, yielding new emerging capabilities. In particular, strong video understanding and cross-scenario capabilities are demonstrated through task transfer from images to videos. |
| title | LLaVA-OneVision: Easy Visual Task Transfer |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2408.03326 |