Towards Universal Video MLLMs with Attribute-Structured and Quality-Verified Instructions
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912903303528448 |
|---|---|
| author | Li, Yunheng Zhang, Hengrui Guo, Meng-Hao Gao, Wenzhao Jia, Shaoyong Jiao, Shaohui Hou, Qibin Cheng, Ming-Ming |
| author_facet | Li, Yunheng Zhang, Hengrui Guo, Meng-Hao Gao, Wenzhao Jia, Shaoyong Jiao, Shaohui Hou, Qibin Cheng, Ming-Ming |
| contents | Universal video understanding requires modeling fine-grained visual and audio information over time in diverse real-world scenarios. However, the performance of existing models is primarily constrained by video-instruction data that represents complex audiovisual content as single, incomplete descriptions, lacking fine-grained organization and reliable annotation. To address this, we introduce: (i) ASID-1M, an open-source collection of one million structured, fine-grained audiovisual instruction annotations with single- and multi-attribute supervision; (ii) ASID-Verify, a scalable data curation pipeline for annotation, with automatic verification and refinement that enforces semantic and temporal consistency between descriptions and the corresponding audiovisual content; and (iii) ASID-Captioner, a video understanding model trained via Supervised Fine-Tuning (SFT) on the ASID-1M. Experiments across seven benchmarks covering audiovisual captioning, attribute-wise captioning, caption-based QA, and caption-based temporal grounding show that ASID-Captioner improves fine-grained caption quality while reducing hallucinations and improving instruction following. It achieves state-of-the-art performance among open-source models and is competitive with Gemini-3-Pro. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_13013 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Towards Universal Video MLLMs with Attribute-Structured and Quality-Verified Instructions Li, Yunheng Zhang, Hengrui Guo, Meng-Hao Gao, Wenzhao Jia, Shaoyong Jiao, Shaohui Hou, Qibin Cheng, Ming-Ming Computer Vision and Pattern Recognition Universal video understanding requires modeling fine-grained visual and audio information over time in diverse real-world scenarios. However, the performance of existing models is primarily constrained by video-instruction data that represents complex audiovisual content as single, incomplete descriptions, lacking fine-grained organization and reliable annotation. To address this, we introduce: (i) ASID-1M, an open-source collection of one million structured, fine-grained audiovisual instruction annotations with single- and multi-attribute supervision; (ii) ASID-Verify, a scalable data curation pipeline for annotation, with automatic verification and refinement that enforces semantic and temporal consistency between descriptions and the corresponding audiovisual content; and (iii) ASID-Captioner, a video understanding model trained via Supervised Fine-Tuning (SFT) on the ASID-1M. Experiments across seven benchmarks covering audiovisual captioning, attribute-wise captioning, caption-based QA, and caption-based temporal grounding show that ASID-Captioner improves fine-grained caption quality while reducing hallucinations and improving instruction following. It achieves state-of-the-art performance among open-source models and is competitive with Gemini-3-Pro. |
| title | Towards Universal Video MLLMs with Attribute-Structured and Quality-Verified Instructions |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2602.13013 |