LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913161737666560 |
|---|---|
| author | An, Xiang Xie, Yin Tang, Feilong Yan, Yunyao Tan, Huajie Zhu, Didi Chen, Changrui Zhao, Xiuwei Qin, Bin Yang, Kaicheng Shen, Yifei Zhang, Yuanhan Zhang, Kaichen Zhang, Wenkang Cheng, Zheng Zhang, Nansen Wu, Chunsheng Ge, Chunjiang Ran, Zimin Song, Dehua Li, Chunyuan Feng, Shikun Hu, Ming Chen, Zhangquan Niu, Junbo Li, Bo Feng, Ziyong Liu, Ziwei Ge, Zongyuan Deng, Jiankang |
| author_facet | An, Xiang Xie, Yin Tang, Feilong Yan, Yunyao Tan, Huajie Zhu, Didi Chen, Changrui Zhao, Xiuwei Qin, Bin Yang, Kaicheng Shen, Yifei Zhang, Yuanhan Zhang, Kaichen Zhang, Wenkang Cheng, Zheng Zhang, Nansen Wu, Chunsheng Ge, Chunjiang Ran, Zimin Song, Dehua Li, Chunyuan Feng, Shikun Hu, Ming Chen, Zhangquan Niu, Junbo Li, Bo Feng, Ziyong Liu, Ziwei Ge, Zongyuan Deng, Jiankang |
| contents | We introduce LLaVA-OneVision-2 (LLaVA-OV-2), the most capable vision-language model in the LLaVA-OneVision series to date, achieving superior performance across a broad range of multimodal benchmarks. The model builds on a native OneVision-Encoder and incorporates Windowed Attention for efficient local computation while maintaining native resolution. Its key advance is codec-stream tokenization: it treats compressed video as a continuous bit-cost stream, where bit-cost dynamics determine adaptive temporal groups, and motion-residual cues select salient spatial evidence into compact visual canvases. This allocation concentrates a limited token budget on event-bearing content, enabling more stable long-video token compression than fixed groups of pictures. A shared 3D RoPE further places codec canvases, sampled frames, and images in a unified spatiotemporal coordinate system. Furthermore, we build the LLaVA-OV-2 data and training stack around large-scale open supervision: approximately 8M re-captioned video samples for pretraining, a 4M-sample spatial corpus for fine-tuning. We also introduce JumpScore, a temporal-localization benchmark targeting fine-grained grounding in high-frequency, densely repeated motion, a regime underrepresented by existing video evaluations. A standout capability of LLaVA-OV-2 is its unified perception across video understanding, temporal grounding, spatial grounding, and manipulation-trace reasoning. On JumpScore, LLaVA-OneVision-2-8B reaches 74.9 JumpScore mAP, surpassing Qwen3-VL-8B (30.1) by +44.8 points; under matched visual-token budgets on the same benchmark, codec-stream inputs improve temporal grounding over frame sampling by +9.7 points. Across standard benchmarks, LLaVA-OneVision-2-8B further outperforms Qwen3-VL-8B by +4.3 average points on video tasks, +5.3 on spatial tasks, and +15.6 average J&F on tracking tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_25979 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence An, Xiang Xie, Yin Tang, Feilong Yan, Yunyao Tan, Huajie Zhu, Didi Chen, Changrui Zhao, Xiuwei Qin, Bin Yang, Kaicheng Shen, Yifei Zhang, Yuanhan Zhang, Kaichen Zhang, Wenkang Cheng, Zheng Zhang, Nansen Wu, Chunsheng Ge, Chunjiang Ran, Zimin Song, Dehua Li, Chunyuan Feng, Shikun Hu, Ming Chen, Zhangquan Niu, Junbo Li, Bo Feng, Ziyong Liu, Ziwei Ge, Zongyuan Deng, Jiankang Computer Vision and Pattern Recognition We introduce LLaVA-OneVision-2 (LLaVA-OV-2), the most capable vision-language model in the LLaVA-OneVision series to date, achieving superior performance across a broad range of multimodal benchmarks. The model builds on a native OneVision-Encoder and incorporates Windowed Attention for efficient local computation while maintaining native resolution. Its key advance is codec-stream tokenization: it treats compressed video as a continuous bit-cost stream, where bit-cost dynamics determine adaptive temporal groups, and motion-residual cues select salient spatial evidence into compact visual canvases. This allocation concentrates a limited token budget on event-bearing content, enabling more stable long-video token compression than fixed groups of pictures. A shared 3D RoPE further places codec canvases, sampled frames, and images in a unified spatiotemporal coordinate system. Furthermore, we build the LLaVA-OV-2 data and training stack around large-scale open supervision: approximately 8M re-captioned video samples for pretraining, a 4M-sample spatial corpus for fine-tuning. We also introduce JumpScore, a temporal-localization benchmark targeting fine-grained grounding in high-frequency, densely repeated motion, a regime underrepresented by existing video evaluations. A standout capability of LLaVA-OV-2 is its unified perception across video understanding, temporal grounding, spatial grounding, and manipulation-trace reasoning. On JumpScore, LLaVA-OneVision-2-8B reaches 74.9 JumpScore mAP, surpassing Qwen3-VL-8B (30.1) by +44.8 points; under matched visual-token budgets on the same benchmark, codec-stream inputs improve temporal grounding over frame sampling by +9.7 points. Across standard benchmarks, LLaVA-OneVision-2-8B further outperforms Qwen3-VL-8B by +4.3 average points on video tasks, +5.3 on spatial tasks, and +15.6 average J&F on tracking tasks. |
| title | LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2605.25979 |