Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2503.13377 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912455425261568 |
|---|---|
| author | Wang, Ye Wang, Ziheng Xu, Boshen Du, Yang Lin, Kejun Xiao, Zihan Yue, Zihao Ju, Jianzhong Zhang, Liang Yang, Dingyi Fang, Xiangnan He, Zewen Luo, Zhenbo Wang, Wenxuan Lin, Junqi Luan, Jian Jin, Qin |
| author_facet | Wang, Ye Wang, Ziheng Xu, Boshen Du, Yang Lin, Kejun Xiao, Zihan Yue, Zihao Ju, Jianzhong Zhang, Liang Yang, Dingyi Fang, Xiangnan He, Zewen Luo, Zhenbo Wang, Wenxuan Lin, Junqi Luan, Jian Jin, Qin |
| contents | Temporal Video Grounding (TVG), the task of locating specific video segments based on language queries, is a core challenge in long-form video understanding. While recent Large Vision-Language Models (LVLMs) have shown early promise in tackling TVG through supervised fine-tuning (SFT), their abilities to generalize remain limited. To address this, we propose a novel post-training framework that enhances the generalization capabilities of LVLMs via reinforcement learning (RL). Specifically, our contributions span three key directions: (1) Time-R1: we introduce a reasoning-guided post-training framework via RL with verifiable reward to enhance the capabilities of LVLMs on the TVG task. (2) TimeRFT: we explore data-efficient post-training strategies on our curated RL-friendly dataset, which trains the model to progressively comprehend difficult samples, leading to better generalization. (3) TVGBench: we carefully construct a small yet comprehensive benchmark for LVLM evaluation, assessing 11 types of queries and featuring balanced distributions across both videos and queries. Extensive experiments demonstrate that Time-R1 achieves state-of-the-art performance across multiple downstream datasets using only 2.5K training data, while improving its general video understanding capabilities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_13377 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding Wang, Ye Wang, Ziheng Xu, Boshen Du, Yang Lin, Kejun Xiao, Zihan Yue, Zihao Ju, Jianzhong Zhang, Liang Yang, Dingyi Fang, Xiangnan He, Zewen Luo, Zhenbo Wang, Wenxuan Lin, Junqi Luan, Jian Jin, Qin Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Temporal Video Grounding (TVG), the task of locating specific video segments based on language queries, is a core challenge in long-form video understanding. While recent Large Vision-Language Models (LVLMs) have shown early promise in tackling TVG through supervised fine-tuning (SFT), their abilities to generalize remain limited. To address this, we propose a novel post-training framework that enhances the generalization capabilities of LVLMs via reinforcement learning (RL). Specifically, our contributions span three key directions: (1) Time-R1: we introduce a reasoning-guided post-training framework via RL with verifiable reward to enhance the capabilities of LVLMs on the TVG task. (2) TimeRFT: we explore data-efficient post-training strategies on our curated RL-friendly dataset, which trains the model to progressively comprehend difficult samples, leading to better generalization. (3) TVGBench: we carefully construct a small yet comprehensive benchmark for LVLM evaluation, assessing 11 types of queries and featuring balanced distributions across both videos and queries. Extensive experiments demonstrate that Time-R1 achieves state-of-the-art performance across multiple downstream datasets using only 2.5K training data, while improving its general video understanding capabilities. |
| title | Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2503.13377 |