MSC: A Marine Wildlife Video Dataset with Grounded Segmentation and Clip-Level Captioning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909762477621248 |
|---|---|
| author | Truong, Quang-Trung Wong, Yuk-Kwan Dang, Vo Hoang Kim Tuyen Gotama, Rinaldi Nguyen, Duc Thanh Yeung, Sai-Kit |
| author_facet | Truong, Quang-Trung Wong, Yuk-Kwan Dang, Vo Hoang Kim Tuyen Gotama, Rinaldi Nguyen, Duc Thanh Yeung, Sai-Kit |
| contents | Marine videos present significant challenges for video understanding due to the dynamics of marine objects and the surrounding environment, camera motion, and the complexity of underwater scenes. Existing video captioning datasets, typically focused on generic or human-centric domains, often fail to generalize to the complexities of the marine environment and gain insights about marine life. To address these limitations, we propose a two-stage marine object-oriented video captioning pipeline. We introduce a comprehensive video understanding benchmark that leverages the triplets of video, text, and segmentation masks to facilitate visual grounding and captioning, leading to improved marine video understanding and analysis, and marine video generation. Additionally, we highlight the effectiveness of video splitting in order to detect salient object transitions in scene changes, which significantly enrich the semantics of captioning content. Our dataset and code have been released at https://msc.hkustvgd.com. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_04549 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MSC: A Marine Wildlife Video Dataset with Grounded Segmentation and Clip-Level Captioning Truong, Quang-Trung Wong, Yuk-Kwan Dang, Vo Hoang Kim Tuyen Gotama, Rinaldi Nguyen, Duc Thanh Yeung, Sai-Kit Computer Vision and Pattern Recognition Artificial Intelligence Multimedia Marine videos present significant challenges for video understanding due to the dynamics of marine objects and the surrounding environment, camera motion, and the complexity of underwater scenes. Existing video captioning datasets, typically focused on generic or human-centric domains, often fail to generalize to the complexities of the marine environment and gain insights about marine life. To address these limitations, we propose a two-stage marine object-oriented video captioning pipeline. We introduce a comprehensive video understanding benchmark that leverages the triplets of video, text, and segmentation masks to facilitate visual grounding and captioning, leading to improved marine video understanding and analysis, and marine video generation. Additionally, we highlight the effectiveness of video splitting in order to detect salient object transitions in scene changes, which significantly enrich the semantics of captioning content. Our dataset and code have been released at https://msc.hkustvgd.com. |
| title | MSC: A Marine Wildlife Video Dataset with Grounded Segmentation and Clip-Level Captioning |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Multimedia |
| url | https://arxiv.org/abs/2508.04549 |