VidCapBench: A Comprehensive Benchmark of Video Captioning for Controllable Text-to-Video Generation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908368121102336 |
|---|---|
| author | Chen, Xinlong Zhang, Yuanxing Rao, Chongling Guan, Yushuo Liu, Jiaheng Zhang, Fuzheng Song, Chengru Liu, Qiang Zhang, Di Tan, Tieniu |
| author_facet | Chen, Xinlong Zhang, Yuanxing Rao, Chongling Guan, Yushuo Liu, Jiaheng Zhang, Fuzheng Song, Chengru Liu, Qiang Zhang, Di Tan, Tieniu |
| contents | The training of controllable text-to-video (T2V) models relies heavily on the alignment between videos and captions, yet little existing research connects video caption evaluation with T2V generation assessment. This paper introduces VidCapBench, a video caption evaluation scheme specifically designed for T2V generation, agnostic to any particular caption format. VidCapBench employs a data annotation pipeline, combining expert model labeling and human refinement, to associate each collected video with key information spanning video aesthetics, content, motion, and physical laws. VidCapBench then partitions these key information attributes into automatically assessable and manually assessable subsets, catering to both the rapid evaluation needs of agile development and the accuracy requirements of thorough validation. By evaluating numerous state-of-the-art captioning models, we demonstrate the superior stability and comprehensiveness of VidCapBench compared to existing video captioning evaluation approaches. Verification with off-the-shelf T2V models reveals a significant positive correlation between scores on VidCapBench and the T2V quality evaluation metrics, indicating that VidCapBench can provide valuable guidance for training T2V models. The project is available at https://github.com/VidCapBench/VidCapBench. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_12782 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VidCapBench: A Comprehensive Benchmark of Video Captioning for Controllable Text-to-Video Generation Chen, Xinlong Zhang, Yuanxing Rao, Chongling Guan, Yushuo Liu, Jiaheng Zhang, Fuzheng Song, Chengru Liu, Qiang Zhang, Di Tan, Tieniu Artificial Intelligence The training of controllable text-to-video (T2V) models relies heavily on the alignment between videos and captions, yet little existing research connects video caption evaluation with T2V generation assessment. This paper introduces VidCapBench, a video caption evaluation scheme specifically designed for T2V generation, agnostic to any particular caption format. VidCapBench employs a data annotation pipeline, combining expert model labeling and human refinement, to associate each collected video with key information spanning video aesthetics, content, motion, and physical laws. VidCapBench then partitions these key information attributes into automatically assessable and manually assessable subsets, catering to both the rapid evaluation needs of agile development and the accuracy requirements of thorough validation. By evaluating numerous state-of-the-art captioning models, we demonstrate the superior stability and comprehensiveness of VidCapBench compared to existing video captioning evaluation approaches. Verification with off-the-shelf T2V models reveals a significant positive correlation between scores on VidCapBench and the T2V quality evaluation metrics, indicating that VidCapBench can provide valuable guidance for training T2V models. The project is available at https://github.com/VidCapBench/VidCapBench. |
| title | VidCapBench: A Comprehensive Benchmark of Video Captioning for Controllable Text-to-Video Generation |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2502.12782 |