An Efficient Streaming Video Understanding Framework with Agentic Control
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917552043589632 |
|---|---|
| author | Liu, Jinming Huang, Jianguo Jia, Zhaoyang Li, Jiahao Zhang, Xiaoyi Guo, Zongyu Li, Bin Zeng, Wenjun Lu, Yan Jin, Xin |
| author_facet | Liu, Jinming Huang, Jianguo Jia, Zhaoyang Li, Jiahao Zhang, Xiaoyi Guo, Zongyu Li, Bin Zeng, Wenjun Lu, Yan Jin, Xin |
| contents | Streaming video requires handling dynamic information density under strict latency budgets. Yet, existing methods typically employ static strategies, such as fixed memory compression or reliance on a single model, forcing a trade-off: fast models fail on complex queries, while always-on heavy models violate real-time constraints and overcomplicate simple queries. Rather than fixing these decisions upfront, we propose R3-Streaming (Remember, Respond, Reason), which formulates streaming video understanding as a cascaded control problem: for each query, the system compresses memory, judges response readiness, and routes computation sequentially, so that each downstream decision builds on progressively refined information states. To optimize this pipeline, we introduce an age-aware forgetting policy for memory compression, as aggressively compressing historical frames can yield substantial performance gains. For compute routing, we propose TB-GRPO, a target-balanced reinforcement learning objective that routes hard queries to a stronger model while preventing mode collapse. Extensive evaluations demonstrate that R3-Streaming achieves state-of-the-art results among streaming MLLMs, reaching 57.92 on OVO-Bench and 76.36 on StreamingBench, while reducing visual token usage by 95 to 96 percent. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_17921 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | An Efficient Streaming Video Understanding Framework with Agentic Control Liu, Jinming Huang, Jianguo Jia, Zhaoyang Li, Jiahao Zhang, Xiaoyi Guo, Zongyu Li, Bin Zeng, Wenjun Lu, Yan Jin, Xin Computer Vision and Pattern Recognition Streaming video requires handling dynamic information density under strict latency budgets. Yet, existing methods typically employ static strategies, such as fixed memory compression or reliance on a single model, forcing a trade-off: fast models fail on complex queries, while always-on heavy models violate real-time constraints and overcomplicate simple queries. Rather than fixing these decisions upfront, we propose R3-Streaming (Remember, Respond, Reason), which formulates streaming video understanding as a cascaded control problem: for each query, the system compresses memory, judges response readiness, and routes computation sequentially, so that each downstream decision builds on progressively refined information states. To optimize this pipeline, we introduce an age-aware forgetting policy for memory compression, as aggressively compressing historical frames can yield substantial performance gains. For compute routing, we propose TB-GRPO, a target-balanced reinforcement learning objective that routes hard queries to a stronger model while preventing mode collapse. Extensive evaluations demonstrate that R3-Streaming achieves state-of-the-art results among streaming MLLMs, reaching 57.92 on OVO-Bench and 76.36 on StreamingBench, while reducing visual token usage by 95 to 96 percent. |
| title | An Efficient Streaming Video Understanding Framework with Agentic Control |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2605.17921 |