Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2504.05541 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908308080689152 |
|---|---|
| author | Tang, Yunlong Bi, Jing Huang, Chao Liang, Susan Shimada, Daiki Hua, Hang Xiao, Yunzhong Song, Yizhi Liu, Pinxin Feng, Mingqian Guo, Junjia Liu, Zhuo Song, Luchuan Vosoughi, Ali He, Jinxi He, Liu Zhang, Zeliang Luo, Jiebo Xu, Chenliang |
| author_facet | Tang, Yunlong Bi, Jing Huang, Chao Liang, Susan Shimada, Daiki Hua, Hang Xiao, Yunzhong Song, Yizhi Liu, Pinxin Feng, Mingqian Guo, Junjia Liu, Zhuo Song, Luchuan Vosoughi, Ali He, Jinxi He, Liu Zhang, Zeliang Luo, Jiebo Xu, Chenliang |
| contents | We present CAT-V (Caption AnyThing in Video), a training-free framework for fine-grained object-centric video captioning that enables detailed descriptions of user-selected objects through time. CAT-V integrates three key components: a Segmenter based on SAMURAI for precise object segmentation across frames, a Temporal Analyzer powered by TRACE-Uni for accurate event boundary detection and temporal analysis, and a Captioner using InternVL-2.5 for generating detailed object-centric descriptions. Through spatiotemporal visual prompts and chain-of-thought reasoning, our framework generates detailed, temporally-aware descriptions of objects' attributes, actions, statuses, interactions, and environmental contexts without requiring additional training data. CAT-V supports flexible user interactions through various visual prompts (points, bounding boxes, and irregular regions) and maintains temporal sensitivity by tracking object states and interactions across different time segments. Our approach addresses limitations of existing video captioning methods, which either produce overly abstract descriptions or lack object-level precision, enabling fine-grained, object-specific descriptions while maintaining temporal coherence and spatial accuracy. The GitHub repository for this project is available at https://github.com/yunlong10/CAT-V |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_05541 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Caption Anything in Video: Fine-grained Object-centric Captioning via Spatiotemporal Multimodal Prompting Tang, Yunlong Bi, Jing Huang, Chao Liang, Susan Shimada, Daiki Hua, Hang Xiao, Yunzhong Song, Yizhi Liu, Pinxin Feng, Mingqian Guo, Junjia Liu, Zhuo Song, Luchuan Vosoughi, Ali He, Jinxi He, Liu Zhang, Zeliang Luo, Jiebo Xu, Chenliang Computer Vision and Pattern Recognition We present CAT-V (Caption AnyThing in Video), a training-free framework for fine-grained object-centric video captioning that enables detailed descriptions of user-selected objects through time. CAT-V integrates three key components: a Segmenter based on SAMURAI for precise object segmentation across frames, a Temporal Analyzer powered by TRACE-Uni for accurate event boundary detection and temporal analysis, and a Captioner using InternVL-2.5 for generating detailed object-centric descriptions. Through spatiotemporal visual prompts and chain-of-thought reasoning, our framework generates detailed, temporally-aware descriptions of objects' attributes, actions, statuses, interactions, and environmental contexts without requiring additional training data. CAT-V supports flexible user interactions through various visual prompts (points, bounding boxes, and irregular regions) and maintains temporal sensitivity by tracking object states and interactions across different time segments. Our approach addresses limitations of existing video captioning methods, which either produce overly abstract descriptions or lack object-level precision, enabling fine-grained, object-specific descriptions while maintaining temporal coherence and spatial accuracy. The GitHub repository for this project is available at https://github.com/yunlong10/CAT-V |
| title | Caption Anything in Video: Fine-grained Object-centric Captioning via Spatiotemporal Multimodal Prompting |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.05541 |