Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.03447 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916042053255168 |
|---|---|
| author | Yan, Weicai Dai, Yuhong Ran, Qi Li, Haodong Lin, Wang Jin, Tao Xie, Xing Liao, Hao Lian, Jianxun |
| author_facet | Yan, Weicai Dai, Yuhong Ran, Qi Li, Haodong Lin, Wang Jin, Tao Xie, Xing Liao, Hao Lian, Jianxun |
| contents | Proactive and real-time interactive experiences are essential for human-like AI companions, yet face three key challenges: (1) achieving low-latency inference under continuous streaming inputs, (2) autonomously deciding when to respond, and (3) controlling both quality and quantity of generated content to meet real-time constraints. In this work, we instantiate AI companions through two gaming scenarios, commentator and guide, selected for their suitability for automatic evaluation. We introduce the Live Gaming Benchmark, a large-scale dataset with three representative scenarios: solo commentary, co-commentary, and user guidance, and present Proact-VL, a general framework that shapes multimodal language models into proactive, real-time interactive agents capable of human-like environment perception and interaction. Extensive experiments show Proact-VL achieves superior response latency and quality while maintaining strong video understanding capabilities, demonstrating its practicality for real-time interactive applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_03447 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Proact-VL: A Proactive VideoLLM for Real-Time AI Companions Yan, Weicai Dai, Yuhong Ran, Qi Li, Haodong Lin, Wang Jin, Tao Xie, Xing Liao, Hao Lian, Jianxun Computer Vision and Pattern Recognition Proactive and real-time interactive experiences are essential for human-like AI companions, yet face three key challenges: (1) achieving low-latency inference under continuous streaming inputs, (2) autonomously deciding when to respond, and (3) controlling both quality and quantity of generated content to meet real-time constraints. In this work, we instantiate AI companions through two gaming scenarios, commentator and guide, selected for their suitability for automatic evaluation. We introduce the Live Gaming Benchmark, a large-scale dataset with three representative scenarios: solo commentary, co-commentary, and user guidance, and present Proact-VL, a general framework that shapes multimodal language models into proactive, real-time interactive agents capable of human-like environment perception and interaction. Extensive experiments show Proact-VL achieves superior response latency and quality while maintaining strong video understanding capabilities, demonstrating its practicality for real-time interactive applications. |
| title | Proact-VL: A Proactive VideoLLM for Real-Time AI Companions |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.03447 |