OmniVideo-R1: Reinforcing Audio-visual Reasoning with Query Intention and Modality Attention
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917275309703168 |
|---|---|
| author | Chen, Zhangquan Tao, Jiale Li, Ruihuang Hu, Yihao Chen, Ruitao Yang, Zhantao Yu, Xinlei Jing, Haodong Zhang, Manyuan Shao, Shuai Wang, Biao Lu, Qinglin Huang, Ruqi |
| author_facet | Chen, Zhangquan Tao, Jiale Li, Ruihuang Hu, Yihao Chen, Ruitao Yang, Zhantao Yu, Xinlei Jing, Haodong Zhang, Manyuan Shao, Shuai Wang, Biao Lu, Qinglin Huang, Ruqi |
| contents | While humans perceive the world through diverse modalities that operate synergistically to support a holistic understanding of their surroundings, existing omnivideo models still face substantial challenges on audio-visual understanding tasks. In this paper, we propose OmniVideo-R1, a novel reinforced framework that improves mixed-modality reasoning. OmniVideo-R1 empowers models to "think with omnimodal cues" by two key strategies: (1) query-intensive grounding based on self-supervised learning paradigms; and (2) modality-attentive fusion built upon contrastive learning paradigms. Extensive experiments on multiple benchmarks demonstrate that OmniVideo-R1 consistently outperforms strong baselines, highlighting its effectiveness and robust generalization capabilities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_05847 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | OmniVideo-R1: Reinforcing Audio-visual Reasoning with Query Intention and Modality Attention Chen, Zhangquan Tao, Jiale Li, Ruihuang Hu, Yihao Chen, Ruitao Yang, Zhantao Yu, Xinlei Jing, Haodong Zhang, Manyuan Shao, Shuai Wang, Biao Lu, Qinglin Huang, Ruqi Artificial Intelligence Computer Vision and Pattern Recognition I.2.10 While humans perceive the world through diverse modalities that operate synergistically to support a holistic understanding of their surroundings, existing omnivideo models still face substantial challenges on audio-visual understanding tasks. In this paper, we propose OmniVideo-R1, a novel reinforced framework that improves mixed-modality reasoning. OmniVideo-R1 empowers models to "think with omnimodal cues" by two key strategies: (1) query-intensive grounding based on self-supervised learning paradigms; and (2) modality-attentive fusion built upon contrastive learning paradigms. Extensive experiments on multiple benchmarks demonstrate that OmniVideo-R1 consistently outperforms strong baselines, highlighting its effectiveness and robust generalization capabilities. |
| title | OmniVideo-R1: Reinforcing Audio-visual Reasoning with Query Intention and Modality Attention |
| topic | Artificial Intelligence Computer Vision and Pattern Recognition I.2.10 |
| url | https://arxiv.org/abs/2602.05847 |