AURA: Always-On Understanding and Real-Time Assistance via Video Streams
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915917482426368 |
|---|---|
| author | Lu, Xudong Bo, Yang Chen, Jinpeng Li, Shuhan Guo, Xintong Guan, Huankang Liu, Fang Xu, Dunyuan Sun, Peiwen Sun, Heyang Liu, Rui Li, Hongsheng |
| author_facet | Lu, Xudong Bo, Yang Chen, Jinpeng Li, Shuhan Guo, Xintong Guan, Huankang Liu, Fang Xu, Dunyuan Sun, Peiwen Sun, Heyang Liu, Rui Li, Hongsheng |
| contents | Video Large Language Models (VideoLLMs) have achieved strong performance on many video understanding tasks, but most existing systems remain offline and are not well-suited for live video streams that require continuous observation and timely response. Recent streaming VideoLLMs have made progress, yet current approaches often rely on decoupled trigger-response pipelines or are limited to captioning-style narration, reducing their effectiveness for open-ended question answering and long-horizon interaction. We propose AURA (Always-On Understanding and Real-Time Assistance), an end-to-end streaming visual interaction framework that enables a unified VideoLLM to continuously process video streams and support both real-time question answering and proactive responses. AURA integrates context management, data construction, training objectives, and deployment optimization for stable long-horizon streaming interaction. It achieves state-of-the-art performance on streaming benchmarks and supports a real-time demo system with ASR and TTS running at 2 FPS on two 80G accelerators. We release the AURA model together with a real-time inference framework to facilitate future research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_04184 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | AURA: Always-On Understanding and Real-Time Assistance via Video Streams Lu, Xudong Bo, Yang Chen, Jinpeng Li, Shuhan Guo, Xintong Guan, Huankang Liu, Fang Xu, Dunyuan Sun, Peiwen Sun, Heyang Liu, Rui Li, Hongsheng Computer Vision and Pattern Recognition Video Large Language Models (VideoLLMs) have achieved strong performance on many video understanding tasks, but most existing systems remain offline and are not well-suited for live video streams that require continuous observation and timely response. Recent streaming VideoLLMs have made progress, yet current approaches often rely on decoupled trigger-response pipelines or are limited to captioning-style narration, reducing their effectiveness for open-ended question answering and long-horizon interaction. We propose AURA (Always-On Understanding and Real-Time Assistance), an end-to-end streaming visual interaction framework that enables a unified VideoLLM to continuously process video streams and support both real-time question answering and proactive responses. AURA integrates context management, data construction, training objectives, and deployment optimization for stable long-horizon streaming interaction. It achieves state-of-the-art performance on streaming benchmarks and supports a real-time demo system with ASR and TTS running at 2 FPS on two 80G accelerators. We release the AURA model together with a real-time inference framework to facilitate future research. |
| title | AURA: Always-On Understanding and Real-Time Assistance via Video Streams |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2604.04184 |