StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistant
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915501904494592 |
|---|---|
| author | Wang, Haibo Feng, Bo Lai, Zhengfeng Xu, Mingze Li, Shiyu Ge, Weifeng Dehghan, Afshin Cao, Meng Huang, Ping |
| author_facet | Wang, Haibo Feng, Bo Lai, Zhengfeng Xu, Mingze Li, Shiyu Ge, Weifeng Dehghan, Afshin Cao, Meng Huang, Ping |
| contents | We present StreamBridge, a simple yet effective framework that seamlessly transforms offline Video-LLMs into streaming-capable models. It addresses two fundamental challenges in adapting existing models into online scenarios: (1) limited capability for multi-turn real-time understanding, and (2) lack of proactive response mechanisms. Specifically, StreamBridge incorporates (1) a memory buffer combined with a round-decayed compression strategy, supporting long-context multi-turn interactions, and (2) a decoupled, lightweight activation model that can be effortlessly integrated into existing Video-LLMs, enabling continuous proactive responses. To further support StreamBridge, we construct Stream-IT, a large-scale dataset tailored for streaming video understanding, featuring interleaved video-text sequences and diverse instruction formats. Extensive experiments show that StreamBridge significantly improves the streaming understanding capabilities of offline Video-LLMs across various tasks, outperforming even proprietary models such as GPT-4o and Gemini 1.5 Pro. Simultaneously, it achieves competitive or superior performance on standard video understanding benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_05467 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistant Wang, Haibo Feng, Bo Lai, Zhengfeng Xu, Mingze Li, Shiyu Ge, Weifeng Dehghan, Afshin Cao, Meng Huang, Ping Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language We present StreamBridge, a simple yet effective framework that seamlessly transforms offline Video-LLMs into streaming-capable models. It addresses two fundamental challenges in adapting existing models into online scenarios: (1) limited capability for multi-turn real-time understanding, and (2) lack of proactive response mechanisms. Specifically, StreamBridge incorporates (1) a memory buffer combined with a round-decayed compression strategy, supporting long-context multi-turn interactions, and (2) a decoupled, lightweight activation model that can be effortlessly integrated into existing Video-LLMs, enabling continuous proactive responses. To further support StreamBridge, we construct Stream-IT, a large-scale dataset tailored for streaming video understanding, featuring interleaved video-text sequences and diverse instruction formats. Extensive experiments show that StreamBridge significantly improves the streaming understanding capabilities of offline Video-LLMs across various tasks, outperforming even proprietary models such as GPT-4o and Gemini 1.5 Pro. Simultaneously, it achieves competitive or superior performance on standard video understanding benchmarks. |
| title | StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistant |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2505.05467 |