The Solution for Temporal Sound Localisation Task of ICCV 1st Perception Test Challenge 2023
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914855955464192 |
|---|---|
| author | Huang, Yurui Yang, Yang Chen, Shou Wu, Xiangyu Chen, Qingguo Lu, Jianfeng |
| author_facet | Huang, Yurui Yang, Yang Chen, Shou Wu, Xiangyu Chen, Qingguo Lu, Jianfeng |
| contents | In this paper, we propose a solution for improving the quality of temporal sound localization. We employ a multimodal fusion approach to combine visual and audio features. High-quality visual features are extracted using a state-of-the-art self-supervised pre-training network, resulting in efficient video feature representations. At the same time, audio features serve as complementary information to help the model better localize the start and end of sounds. The fused features are trained in a multi-scale Transformer for training. In the final test dataset, we achieved a mean average precision (mAP) of 0.33, obtaining the second-best performance in this track. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_02318 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The Solution for Temporal Sound Localisation Task of ICCV 1st Perception Test Challenge 2023 Huang, Yurui Yang, Yang Chen, Shou Wu, Xiangyu Chen, Qingguo Lu, Jianfeng Sound Computer Vision and Pattern Recognition Machine Learning Audio and Speech Processing In this paper, we propose a solution for improving the quality of temporal sound localization. We employ a multimodal fusion approach to combine visual and audio features. High-quality visual features are extracted using a state-of-the-art self-supervised pre-training network, resulting in efficient video feature representations. At the same time, audio features serve as complementary information to help the model better localize the start and end of sounds. The fused features are trained in a multi-scale Transformer for training. In the final test dataset, we achieved a mean average precision (mAP) of 0.33, obtaining the second-best performance in this track. |
| title | The Solution for Temporal Sound Localisation Task of ICCV 1st Perception Test Challenge 2023 |
| topic | Sound Computer Vision and Pattern Recognition Machine Learning Audio and Speech Processing |
| url | https://arxiv.org/abs/2407.02318 |