Da Yu: Towards USV-Based Image Captioning for Waterway Surveillance and Scene Understanding
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912458719887360 |
|---|---|
| author | Guan, Runwei Ouyang, Ningwei Xu, Tianhao Liang, Shaofeng Dai, Wei Sun, Yafeng Gao, Shang Lai, Songning Yao, Shanliang Hu, Xuming Liu, Ryan Wen Yue, Yutao Xiong, Hui |
| author_facet | Guan, Runwei Ouyang, Ningwei Xu, Tianhao Liang, Shaofeng Dai, Wei Sun, Yafeng Gao, Shang Lai, Songning Yao, Shanliang Hu, Xuming Liu, Ryan Wen Yue, Yutao Xiong, Hui |
| contents | Automated waterway environment perception is crucial for enabling unmanned surface vessels (USVs) to understand their surroundings and make informed decisions. Most existing waterway perception models primarily focus on instance-level object perception paradigms (e.g., detection, segmentation). However, due to the complexity of waterway environments, current perception datasets and models fail to achieve global semantic understanding of waterways, limiting large-scale monitoring and structured log generation. With the advancement of vision-language models (VLMs), we leverage image captioning to introduce WaterCaption, the first captioning dataset specifically designed for waterway environments. WaterCaption focuses on fine-grained, multi-region long-text descriptions, providing a new research direction for visual geo-understanding and spatial scene cognition. Exactly, it includes 20.2k image-text pair data with 1.8 million vocabulary size. Additionally, we propose Da Yu, an edge-deployable multi-modal large language model for USVs, where we propose a novel vision-to-language projector called Nano Transformer Adaptor (NTA). NTA effectively balances computational efficiency with the capacity for both global and fine-grained local modeling of visual features, thereby significantly enhancing the model's ability to generate long-form textual outputs. Da Yu achieves an optimal balance between performance and efficiency, surpassing state-of-the-art models on WaterCaption and several other captioning benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_19288 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Da Yu: Towards USV-Based Image Captioning for Waterway Surveillance and Scene Understanding Guan, Runwei Ouyang, Ningwei Xu, Tianhao Liang, Shaofeng Dai, Wei Sun, Yafeng Gao, Shang Lai, Songning Yao, Shanliang Hu, Xuming Liu, Ryan Wen Yue, Yutao Xiong, Hui Computer Vision and Pattern Recognition Robotics Automated waterway environment perception is crucial for enabling unmanned surface vessels (USVs) to understand their surroundings and make informed decisions. Most existing waterway perception models primarily focus on instance-level object perception paradigms (e.g., detection, segmentation). However, due to the complexity of waterway environments, current perception datasets and models fail to achieve global semantic understanding of waterways, limiting large-scale monitoring and structured log generation. With the advancement of vision-language models (VLMs), we leverage image captioning to introduce WaterCaption, the first captioning dataset specifically designed for waterway environments. WaterCaption focuses on fine-grained, multi-region long-text descriptions, providing a new research direction for visual geo-understanding and spatial scene cognition. Exactly, it includes 20.2k image-text pair data with 1.8 million vocabulary size. Additionally, we propose Da Yu, an edge-deployable multi-modal large language model for USVs, where we propose a novel vision-to-language projector called Nano Transformer Adaptor (NTA). NTA effectively balances computational efficiency with the capacity for both global and fine-grained local modeling of visual features, thereby significantly enhancing the model's ability to generate long-form textual outputs. Da Yu achieves an optimal balance between performance and efficiency, surpassing state-of-the-art models on WaterCaption and several other captioning benchmarks. |
| title | Da Yu: Towards USV-Based Image Captioning for Waterway Surveillance and Scene Understanding |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2506.19288 |