GIFT: Global Irreplaceability Frame Targeting for Efficient Video Understanding
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910074054639616 |
|---|---|
| author | Ma, Junpeng Zhou, Sashuai Li, Guanghao Gao, Xin Cao, Yue Zeng, Hengyu Yan, Yuxiang Wang, Zhibin Song, Jun Zheng, Bo Zhang, Shanghang Pu, Jian |
| author_facet | Ma, Junpeng Zhou, Sashuai Li, Guanghao Gao, Xin Cao, Yue Zeng, Hengyu Yan, Yuxiang Wang, Zhibin Song, Jun Zheng, Bo Zhang, Shanghang Pu, Jian |
| contents | Video Large Language Models (VLMs) have achieved remarkable success in video understanding, but the significant computational cost from processing dense frames severely limits their practical application. Existing methods alleviate this by selecting keyframes, but their greedy decision-making, combined with a decoupled evaluation of relevance and diversity, often falls into local optima and results in erroneously selecting irrelevant noise frames. To address these challenges, we propose GIFT: Global Irreplaceability Frame Targeting, a novel training-free framework that selects frames by assessing their intrinsic irreplaceability. Specifically, we first introduce Directed Diversity to quantify a frame's uniqueness conditioned on relevance, which allows us to formulate a unified irreplaceability score. Subsequently, our Budget-Aware Refinement strategy employs a adaptive iterative process that first secures a core set of frames with the highest irreplaceability, and then shifts its priority to building crucial temporal context around these selections as the budget expands. Extensive experiments demonstrate that GIFT achieves a maximum average improvement of 12.5% across long-form video benchmarks on LLaVA-Video-7B compared to uniform sampling. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_25072 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | GIFT: Global Irreplaceability Frame Targeting for Efficient Video Understanding Ma, Junpeng Zhou, Sashuai Li, Guanghao Gao, Xin Cao, Yue Zeng, Hengyu Yan, Yuxiang Wang, Zhibin Song, Jun Zheng, Bo Zhang, Shanghang Pu, Jian Computer Vision and Pattern Recognition Video Large Language Models (VLMs) have achieved remarkable success in video understanding, but the significant computational cost from processing dense frames severely limits their practical application. Existing methods alleviate this by selecting keyframes, but their greedy decision-making, combined with a decoupled evaluation of relevance and diversity, often falls into local optima and results in erroneously selecting irrelevant noise frames. To address these challenges, we propose GIFT: Global Irreplaceability Frame Targeting, a novel training-free framework that selects frames by assessing their intrinsic irreplaceability. Specifically, we first introduce Directed Diversity to quantify a frame's uniqueness conditioned on relevance, which allows us to formulate a unified irreplaceability score. Subsequently, our Budget-Aware Refinement strategy employs a adaptive iterative process that first secures a core set of frames with the highest irreplaceability, and then shifts its priority to building crucial temporal context around these selections as the budget expands. Extensive experiments demonstrate that GIFT achieves a maximum average improvement of 12.5% across long-form video benchmarks on LLaVA-Video-7B compared to uniform sampling. |
| title | GIFT: Global Irreplaceability Frame Targeting for Efficient Video Understanding |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.25072 |