Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.13707 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909540860035072 |
|---|---|
| author | Gurukar, Saket Kadav, Asim |
| author_facet | Gurukar, Saket Kadav, Asim |
| contents | Long-form video understanding is essential for various applications such as video retrieval, summarizing, and question answering. Yet, traditional approaches demand substantial computing power and are often bottlenecked by GPU memory. To tackle this issue, we present Long-Video Memory Network, Long-VMNet, a novel video understanding method that employs a fixed-size memory representation to store discriminative patches sampled from the input video. Long-VMNet achieves improved efficiency by leveraging a neural sampler that identifies discriminative tokens. Additionally, Long-VMNet only needs one scan through the video, greatly boosting efficiency. Our results on the Rest-ADL dataset demonstrate an 18x -- 75x improvement in inference times for long-form video retrieval and answering questions, with a competitive predictive performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_13707 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Long-VMNet: Accelerating Long-Form Video Understanding via Fixed Memory Gurukar, Saket Kadav, Asim Computer Vision and Pattern Recognition Long-form video understanding is essential for various applications such as video retrieval, summarizing, and question answering. Yet, traditional approaches demand substantial computing power and are often bottlenecked by GPU memory. To tackle this issue, we present Long-Video Memory Network, Long-VMNet, a novel video understanding method that employs a fixed-size memory representation to store discriminative patches sampled from the input video. Long-VMNet achieves improved efficiency by leveraging a neural sampler that identifies discriminative tokens. Additionally, Long-VMNet only needs one scan through the video, greatly boosting efficiency. Our results on the Rest-ADL dataset demonstrate an 18x -- 75x improvement in inference times for long-form video retrieval and answering questions, with a competitive predictive performance. |
| title | Long-VMNet: Accelerating Long-Form Video Understanding via Fixed Memory |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2503.13707 |