Multi-Microphone Speech Emotion Recognition using the Hierarchical Token-semantic Audio Transformer Architecture
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912027966963712 |
|---|---|
| author | Cohen, Ohad Hazan, Gershon Gannot, Sharon |
| author_facet | Cohen, Ohad Hazan, Gershon Gannot, Sharon |
| contents | The performance of most emotion recognition systems degrades in real-life situations ('in the wild' scenarios) where the audio is contaminated by reverberation. Our study explores new methods to alleviate the performance degradation of SER algorithms and develop a more robust system for adverse conditions. We propose processing multi-microphone signals to address these challenges and improve emotion classification accuracy. We adopt a state-of-the-art transformer model, the HTS-AT, to handle multi-channel audio inputs. We evaluate two strategies: averaging mel-spectrograms across channels and summing patch-embedded representations. Our multi-microphone model achieves superior performance compared to single-channel baselines when tested on real-world reverberant environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_03272 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Multi-Microphone Speech Emotion Recognition using the Hierarchical Token-semantic Audio Transformer Architecture Cohen, Ohad Hazan, Gershon Gannot, Sharon Audio and Speech Processing Artificial Intelligence Machine Learning The performance of most emotion recognition systems degrades in real-life situations ('in the wild' scenarios) where the audio is contaminated by reverberation. Our study explores new methods to alleviate the performance degradation of SER algorithms and develop a more robust system for adverse conditions. We propose processing multi-microphone signals to address these challenges and improve emotion classification accuracy. We adopt a state-of-the-art transformer model, the HTS-AT, to handle multi-channel audio inputs. We evaluate two strategies: averaging mel-spectrograms across channels and summing patch-embedded representations. Our multi-microphone model achieves superior performance compared to single-channel baselines when tested on real-world reverberant environments. |
| title | Multi-Microphone Speech Emotion Recognition using the Hierarchical Token-semantic Audio Transformer Architecture |
| topic | Audio and Speech Processing Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2406.03272 |