FiLA-Video: Spatio-Temporal Compression for Fine-Grained Long Video Understanding
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912353038106624 |
|---|---|
| author | Guo, Yanan Dong, Wenhui Song, Jun Zhu, Shiding Zhang, Xuan Yang, Hanqing Wang, Yingbo Du, Yang Chen, Xianing Zheng, Bo |
| author_facet | Guo, Yanan Dong, Wenhui Song, Jun Zhu, Shiding Zhang, Xuan Yang, Hanqing Wang, Yingbo Du, Yang Chen, Xianing Zheng, Bo |
| contents | Recent advancements in video understanding within visual large language models (VLLMs) have led to notable progress. However, the complexity of video data and contextual processing limitations still hinder long-video comprehension. A common approach is video feature compression to reduce token input to large language models, yet many methods either fail to prioritize essential features, leading to redundant inter-frame information, or introduce computationally expensive modules.To address these issues, we propose FiLA(Fine-grained Vision Language Model)-Video, a novel framework that leverages a lightweight dynamic-weight multi-frame fusion strategy, which adaptively integrates multiple frames into a single representation while preserving key video information and reducing computational costs. To enhance frame selection for fusion, we introduce a keyframe selection strategy, effectively identifying informative frames from a larger pool for improved summarization. Additionally, we present a simple yet effective long-video training data generation strategy, boosting model performance without extensive manual annotation. Experimental results demonstrate that FiLA-Video achieves superior efficiency and accuracy in long-video comprehension compared to existing methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_20384 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FiLA-Video: Spatio-Temporal Compression for Fine-Grained Long Video Understanding Guo, Yanan Dong, Wenhui Song, Jun Zhu, Shiding Zhang, Xuan Yang, Hanqing Wang, Yingbo Du, Yang Chen, Xianing Zheng, Bo Computer Vision and Pattern Recognition Recent advancements in video understanding within visual large language models (VLLMs) have led to notable progress. However, the complexity of video data and contextual processing limitations still hinder long-video comprehension. A common approach is video feature compression to reduce token input to large language models, yet many methods either fail to prioritize essential features, leading to redundant inter-frame information, or introduce computationally expensive modules.To address these issues, we propose FiLA(Fine-grained Vision Language Model)-Video, a novel framework that leverages a lightweight dynamic-weight multi-frame fusion strategy, which adaptively integrates multiple frames into a single representation while preserving key video information and reducing computational costs. To enhance frame selection for fusion, we introduce a keyframe selection strategy, effectively identifying informative frames from a larger pool for improved summarization. Additionally, we present a simple yet effective long-video training data generation strategy, boosting model performance without extensive manual annotation. Experimental results demonstrate that FiLA-Video achieves superior efficiency and accuracy in long-video comprehension compared to existing methods. |
| title | FiLA-Video: Spatio-Temporal Compression for Fine-Grained Long Video Understanding |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.20384 |