APB-V: Accelerating Long-Video Understanding via Sequence-Parallelism-aware Approximate Attention
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911737617317888 |
|---|---|
| author | Huang, Yuxiang Li, Mingye Han, Xu Xiao, Chaojun Zhao, Weilin Sun, Ao Yuan, Ziqi Zhou, Hao Meng, Fandong Liu, Zhiyuan |
| author_facet | Huang, Yuxiang Li, Mingye Han, Xu Xiao, Chaojun Zhao, Weilin Sun, Ao Yuan, Ziqi Zhou, Hao Meng, Fandong Liu, Zhiyuan |
| contents | The efficiency of long-video inference remains a critical bottleneck, mainly due to the dense computation in the prefill stage of Large Multimodal Models (LMMs). Existing methods either compress visual embeddings or apply sparse attention on a single GPU, yielding limited acceleration or degraded performance and restricting LMMs from handling longer, more complex videos. To overcome these issues, we propose APB-V, a sequence-parallel framework with optimized attention that accelerates long-video inference across multiple GPUs. By distributing approximate attention, APB-V reduces computation and increases parallelism, enabling efficient processing of more visual embeddings without compression and thereby improving task performance. System-level optimizations, such as load balancing and fused forward passes, further unleash the potential of APB-V, delivering speedups of 12.72x, 1.70x, and 1.18x over FlashAttn, ZigZagRing, and APB, without notable performance loss. Code available at https://github.com/thunlp/APB |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_21444 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | APB-V: Accelerating Long-Video Understanding via Sequence-Parallelism-aware Approximate Attention Huang, Yuxiang Li, Mingye Han, Xu Xiao, Chaojun Zhao, Weilin Sun, Ao Yuan, Ziqi Zhou, Hao Meng, Fandong Liu, Zhiyuan Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language The efficiency of long-video inference remains a critical bottleneck, mainly due to the dense computation in the prefill stage of Large Multimodal Models (LMMs). Existing methods either compress visual embeddings or apply sparse attention on a single GPU, yielding limited acceleration or degraded performance and restricting LMMs from handling longer, more complex videos. To overcome these issues, we propose APB-V, a sequence-parallel framework with optimized attention that accelerates long-video inference across multiple GPUs. By distributing approximate attention, APB-V reduces computation and increases parallelism, enabling efficient processing of more visual embeddings without compression and thereby improving task performance. System-level optimizations, such as load balancing and fused forward passes, further unleash the potential of APB-V, delivering speedups of 12.72x, 1.70x, and 1.18x over FlashAttn, ZigZagRing, and APB, without notable performance loss. Code available at https://github.com/thunlp/APB |
| title | APB-V: Accelerating Long-Video Understanding via Sequence-Parallelism-aware Approximate Attention |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2601.21444 |