ESPnet-SpeechLM: An Open Speech Language Model Toolkit
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917935033876480 |
|---|---|
| author | Tian, Jinchuan Shi, Jiatong Chen, William Arora, Siddhant Masuyama, Yoshiki Maekaku, Takashi Wu, Yihan Peng, Junyi Bharadwaj, Shikhar Zhao, Yiwen Cornell, Samuele Peng, Yifan Yue, Xiang Yang, Chao-Han Huck Neubig, Graham Watanabe, Shinji |
| author_facet | Tian, Jinchuan Shi, Jiatong Chen, William Arora, Siddhant Masuyama, Yoshiki Maekaku, Takashi Wu, Yihan Peng, Junyi Bharadwaj, Shikhar Zhao, Yiwen Cornell, Samuele Peng, Yifan Yue, Xiang Yang, Chao-Han Huck Neubig, Graham Watanabe, Shinji |
| contents | We present ESPnet-SpeechLM, an open toolkit designed to democratize the development of speech language models (SpeechLMs) and voice-driven agentic applications. The toolkit standardizes speech processing tasks by framing them as universal sequential modeling problems, encompassing a cohesive workflow of data preprocessing, pre-training, inference, and task evaluation. With ESPnet-SpeechLM, users can easily define task templates and configure key settings, enabling seamless and streamlined SpeechLM development. The toolkit ensures flexibility, efficiency, and scalability by offering highly configurable modules for every stage of the workflow. To illustrate its capabilities, we provide multiple use cases demonstrating how competitive SpeechLMs can be constructed with ESPnet-SpeechLM, including a 1.7B-parameter model pre-trained on both text and speech tasks, across diverse benchmarks. The toolkit and its recipes are fully transparent and reproducible at: https://github.com/espnet/espnet/tree/speechlm. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_15218 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ESPnet-SpeechLM: An Open Speech Language Model Toolkit Tian, Jinchuan Shi, Jiatong Chen, William Arora, Siddhant Masuyama, Yoshiki Maekaku, Takashi Wu, Yihan Peng, Junyi Bharadwaj, Shikhar Zhao, Yiwen Cornell, Samuele Peng, Yifan Yue, Xiang Yang, Chao-Han Huck Neubig, Graham Watanabe, Shinji Computation and Language Sound Audio and Speech Processing We present ESPnet-SpeechLM, an open toolkit designed to democratize the development of speech language models (SpeechLMs) and voice-driven agentic applications. The toolkit standardizes speech processing tasks by framing them as universal sequential modeling problems, encompassing a cohesive workflow of data preprocessing, pre-training, inference, and task evaluation. With ESPnet-SpeechLM, users can easily define task templates and configure key settings, enabling seamless and streamlined SpeechLM development. The toolkit ensures flexibility, efficiency, and scalability by offering highly configurable modules for every stage of the workflow. To illustrate its capabilities, we provide multiple use cases demonstrating how competitive SpeechLMs can be constructed with ESPnet-SpeechLM, including a 1.7B-parameter model pre-trained on both text and speech tasks, across diverse benchmarks. The toolkit and its recipes are fully transparent and reproducible at: https://github.com/espnet/espnet/tree/speechlm. |
| title | ESPnet-SpeechLM: An Open Speech Language Model Toolkit |
| topic | Computation and Language Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2502.15218 |