Interaural time difference loss for binaural target sound extraction
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913455153348608 |
|---|---|
| author | Hernandez-Olivan, Carlos Delcroix, Marc Ochiai, Tsubasa Tawara, Naohiro Nakatani, Tomohiro Araki, Shoko |
| author_facet | Hernandez-Olivan, Carlos Delcroix, Marc Ochiai, Tsubasa Tawara, Naohiro Nakatani, Tomohiro Araki, Shoko |
| contents | Binaural target sound extraction (TSE) aims to extract a desired sound from a binaural mixture of arbitrary sounds while preserving the spatial cues of the desired sound. Indeed, for many applications, the target sound signal and its spatial cues carry important information about the sound source. Binaural TSE can be realized with a neural network trained to output only the desired sound given a binaural mixture and an embedding characterizing the desired sound class as inputs. Conventional TSE systems are trained using signal-level losses, which measure the difference between the extracted and reference signals for the left and right channels. In this paper, we propose adding explicit spatial losses to better preserve the spatial cues of the target sound. In particular, we explore losses aiming at preserving the interaural level (ILD), phase (IPD), and time differences (ITD). We show experimentally that adding such spatial losses, particularly our newly proposed ITD loss, helps preserve better spatial cues while maintaining the signal-level metrics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_00344 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Interaural time difference loss for binaural target sound extraction Hernandez-Olivan, Carlos Delcroix, Marc Ochiai, Tsubasa Tawara, Naohiro Nakatani, Tomohiro Araki, Shoko Sound Audio and Speech Processing Binaural target sound extraction (TSE) aims to extract a desired sound from a binaural mixture of arbitrary sounds while preserving the spatial cues of the desired sound. Indeed, for many applications, the target sound signal and its spatial cues carry important information about the sound source. Binaural TSE can be realized with a neural network trained to output only the desired sound given a binaural mixture and an embedding characterizing the desired sound class as inputs. Conventional TSE systems are trained using signal-level losses, which measure the difference between the extracted and reference signals for the left and right channels. In this paper, we propose adding explicit spatial losses to better preserve the spatial cues of the target sound. In particular, we explore losses aiming at preserving the interaural level (ILD), phase (IPD), and time differences (ITD). We show experimentally that adding such spatial losses, particularly our newly proposed ITD loss, helps preserve better spatial cues while maintaining the signal-level metrics. |
| title | Interaural time difference loss for binaural target sound extraction |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2408.00344 |