Double Mixture: Towards Continual Event Detection from Speech
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910671849914368 |
|---|---|
| author | Kang, Jingqi Wu, Tongtong Zhao, Jinming Wang, Guitao Wei, Yinwei Yang, Hao Qi, Guilin Li, Yuan-Fang Haffari, Gholamreza |
| author_facet | Kang, Jingqi Wu, Tongtong Zhao, Jinming Wang, Guitao Wei, Yinwei Yang, Hao Qi, Guilin Li, Yuan-Fang Haffari, Gholamreza |
| contents | Speech event detection is crucial for multimedia retrieval, involving the tagging of both semantic and acoustic events. Traditional ASR systems often overlook the interplay between these events, focusing solely on content, even though the interpretation of dialogue can vary with environmental context. This paper tackles two primary challenges in speech event detection: the continual integration of new events without forgetting previous ones, and the disentanglement of semantic from acoustic events. We introduce a new task, continual event detection from speech, for which we also provide two benchmark datasets. To address the challenges of catastrophic forgetting and effective disentanglement, we propose a novel method, 'Double Mixture.' This method merges speech expertise with robust memory mechanisms to enhance adaptability and prevent forgetting. Our comprehensive experiments show that this task presents significant challenges that are not effectively addressed by current state-of-the-art methods in either computer vision or natural language processing. Our approach achieves the lowest rates of forgetting and the highest levels of generalization, proving robust across various continual learning sequences. Our code and data are available at https://anonymous.4open.science/status/Continual-SpeechED-6461. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_13289 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Double Mixture: Towards Continual Event Detection from Speech Kang, Jingqi Wu, Tongtong Zhao, Jinming Wang, Guitao Wei, Yinwei Yang, Hao Qi, Guilin Li, Yuan-Fang Haffari, Gholamreza Computation and Language Multimedia Sound Audio and Speech Processing Speech event detection is crucial for multimedia retrieval, involving the tagging of both semantic and acoustic events. Traditional ASR systems often overlook the interplay between these events, focusing solely on content, even though the interpretation of dialogue can vary with environmental context. This paper tackles two primary challenges in speech event detection: the continual integration of new events without forgetting previous ones, and the disentanglement of semantic from acoustic events. We introduce a new task, continual event detection from speech, for which we also provide two benchmark datasets. To address the challenges of catastrophic forgetting and effective disentanglement, we propose a novel method, 'Double Mixture.' This method merges speech expertise with robust memory mechanisms to enhance adaptability and prevent forgetting. Our comprehensive experiments show that this task presents significant challenges that are not effectively addressed by current state-of-the-art methods in either computer vision or natural language processing. Our approach achieves the lowest rates of forgetting and the highest levels of generalization, proving robust across various continual learning sequences. Our code and data are available at https://anonymous.4open.science/status/Continual-SpeechED-6461. |
| title | Double Mixture: Towards Continual Event Detection from Speech |
| topic | Computation and Language Multimedia Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2404.13289 |