LuSeeL: Language-queried Binaural Universal Sound Event Extraction and Localization
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910001889542144 |
|---|---|
| author | Pan, Zexu Zhao, Shengkui Ma, Yukun Wang, Haoxu Jiang, Yiheng Tian, Biao Ma, Bin |
| author_facet | Pan, Zexu Zhao, Shengkui Ma, Yukun Wang, Haoxu Jiang, Yiheng Tian, Biao Ma, Bin |
| contents | Most universal sound extraction algorithms focus on isolating a target sound event from single-channel audio mixtures. However, the real world is three-dimensional, and binaural audio, which mimics human hearing, can capture richer spatial information, including sound source location. This spatial context is crucial for understanding and modeling complex auditory scenes, as it inherently informs sound detection and extraction. In this work, we propose a language-driven universal sound extraction network that isolates text-described sound events from binaural mixtures by effectively leveraging the spatial cues present in binaural signals. Additionally, we jointly predict the direction of arrival (DoA) of the target sound using spatial features from the extraction network. This dual-task approach exploits complementary location information to improve extraction performance while enabling accurate DoA estimation. Experimental results on the in-the-wild AudioCaps dataset show that our proposed LuSeeL model significantly outperforms single-channel and uni-task baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_19153 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | LuSeeL: Language-queried Binaural Universal Sound Event Extraction and Localization Pan, Zexu Zhao, Shengkui Ma, Yukun Wang, Haoxu Jiang, Yiheng Tian, Biao Ma, Bin Audio and Speech Processing Most universal sound extraction algorithms focus on isolating a target sound event from single-channel audio mixtures. However, the real world is three-dimensional, and binaural audio, which mimics human hearing, can capture richer spatial information, including sound source location. This spatial context is crucial for understanding and modeling complex auditory scenes, as it inherently informs sound detection and extraction. In this work, we propose a language-driven universal sound extraction network that isolates text-described sound events from binaural mixtures by effectively leveraging the spatial cues present in binaural signals. Additionally, we jointly predict the direction of arrival (DoA) of the target sound using spatial features from the extraction network. This dual-task approach exploits complementary location information to improve extraction performance while enabling accurate DoA estimation. Experimental results on the in-the-wild AudioCaps dataset show that our proposed LuSeeL model significantly outperforms single-channel and uni-task baselines. |
| title | LuSeeL: Language-queried Binaural Universal Sound Event Extraction and Localization |
| topic | Audio and Speech Processing |
| url | https://arxiv.org/abs/2601.19153 |