Datasets and Recipes for Video Temporal Grounding via Reinforcement Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908464107749376 |
|---|---|
| author | Chen, Ruizhe Fan, Zhiting Luo, Tianze Zou, Heqing Feng, Zhaopeng Xie, Guiyang Zhang, Hansheng Wang, Zhuochen Liu, Zuozhu Zhang, Huaijian |
| author_facet | Chen, Ruizhe Fan, Zhiting Luo, Tianze Zou, Heqing Feng, Zhaopeng Xie, Guiyang Zhang, Hansheng Wang, Zhuochen Liu, Zuozhu Zhang, Huaijian |
| contents | Video Temporal Grounding (VTG) aims to localize relevant temporal segments in videos given natural language queries. Despite recent progress with large vision-language models (LVLMs) and instruction-tuning, existing approaches often suffer from limited temporal awareness and poor generalization. In this work, we introduce a two-stage training framework that integrates supervised fine-tuning with reinforcement learning (RL) to improve both the accuracy and robustness of VTG models. Our approach first leverages high-quality curated cold start data for SFT initialization, followed by difficulty-controlled RL to further enhance temporal localization and reasoning abilities. Comprehensive experiments on multiple VTG benchmarks demonstrate that our method consistently outperforms existing models, particularly in challenging and open-domain scenarios. We conduct an in-depth analysis of training strategies and dataset curation, highlighting the importance of both high-quality cold start data and difficulty-controlled RL. To facilitate further research and industrial adoption, we release all intermediate datasets, models, and code to the community. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_18100 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Datasets and Recipes for Video Temporal Grounding via Reinforcement Learning Chen, Ruizhe Fan, Zhiting Luo, Tianze Zou, Heqing Feng, Zhaopeng Xie, Guiyang Zhang, Hansheng Wang, Zhuochen Liu, Zuozhu Zhang, Huaijian Computer Vision and Pattern Recognition Artificial Intelligence Video Temporal Grounding (VTG) aims to localize relevant temporal segments in videos given natural language queries. Despite recent progress with large vision-language models (LVLMs) and instruction-tuning, existing approaches often suffer from limited temporal awareness and poor generalization. In this work, we introduce a two-stage training framework that integrates supervised fine-tuning with reinforcement learning (RL) to improve both the accuracy and robustness of VTG models. Our approach first leverages high-quality curated cold start data for SFT initialization, followed by difficulty-controlled RL to further enhance temporal localization and reasoning abilities. Comprehensive experiments on multiple VTG benchmarks demonstrate that our method consistently outperforms existing models, particularly in challenging and open-domain scenarios. We conduct an in-depth analysis of training strategies and dataset curation, highlighting the importance of both high-quality cold start data and difficulty-controlled RL. To facilitate further research and industrial adoption, we release all intermediate datasets, models, and code to the community. |
| title | Datasets and Recipes for Video Temporal Grounding via Reinforcement Learning |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2507.18100 |