An Adapter-Based Unified Model for Multiple Spoken Language Processing Tasks
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929393462411264 |
|---|---|
| author | Suresh, Varsha Aït-Mokhtar, Salah Brun, Caroline Calapodescu, Ioan |
| author_facet | Suresh, Varsha Aït-Mokhtar, Salah Brun, Caroline Calapodescu, Ioan |
| contents | Self-supervised learning models have revolutionized the field of speech processing. However, the process of fine-tuning these models on downstream tasks requires substantial computational resources, particularly when dealing with multiple speech-processing tasks. In this paper, we explore the potential of adapter-based fine-tuning in developing a unified model capable of effectively handling multiple spoken language processing tasks. The tasks we investigate are Automatic Speech Recognition, Phoneme Recognition, Intent Classification, Slot Filling, and Spoken Emotion Recognition. We validate our approach through a series of experiments on the SUPERB benchmark, and our results indicate that adapter-based fine-tuning enables a single encoder-decoder model to perform multiple speech processing tasks with an average improvement of 18.4% across the five target tasks while staying efficient in terms of parameter updates. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_14747 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | An Adapter-Based Unified Model for Multiple Spoken Language Processing Tasks Suresh, Varsha Aït-Mokhtar, Salah Brun, Caroline Calapodescu, Ioan Computation and Language Artificial Intelligence Self-supervised learning models have revolutionized the field of speech processing. However, the process of fine-tuning these models on downstream tasks requires substantial computational resources, particularly when dealing with multiple speech-processing tasks. In this paper, we explore the potential of adapter-based fine-tuning in developing a unified model capable of effectively handling multiple spoken language processing tasks. The tasks we investigate are Automatic Speech Recognition, Phoneme Recognition, Intent Classification, Slot Filling, and Spoken Emotion Recognition. We validate our approach through a series of experiments on the SUPERB benchmark, and our results indicate that adapter-based fine-tuning enables a single encoder-decoder model to perform multiple speech processing tasks with an average improvement of 18.4% across the five target tasks while staying efficient in terms of parameter updates. |
| title | An Adapter-Based Unified Model for Multiple Spoken Language Processing Tasks |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2406.14747 |