Building a Taiwanese Mandarin Spoken Language Model: A First Attempt
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910764036521984 |
|---|---|
| author | Yang, Chih-Kai Fu, Yu-Kuan Li, Chen-An Lin, Yi-Cheng Lin, Yu-Xiang Chen, Wei-Chih Chung, Ho Lam Kuan, Chun-Yi Huang, Wei-Ping Lu, Ke-Han Lin, Tzu-Quan Wang, Hsiu-Hsuan Hu, En-Pei Hsu, Chan-Jan Tseng, Liang-Hsuan Chiu, I-Hsiang Sanga, Ulin Chen, Xuanjun Hsu, Po-chun Yang, Shu-wen Lee, Hung-yi |
| author_facet | Yang, Chih-Kai Fu, Yu-Kuan Li, Chen-An Lin, Yi-Cheng Lin, Yu-Xiang Chen, Wei-Chih Chung, Ho Lam Kuan, Chun-Yi Huang, Wei-Ping Lu, Ke-Han Lin, Tzu-Quan Wang, Hsiu-Hsuan Hu, En-Pei Hsu, Chan-Jan Tseng, Liang-Hsuan Chiu, I-Hsiang Sanga, Ulin Chen, Xuanjun Hsu, Po-chun Yang, Shu-wen Lee, Hung-yi |
| contents | This technical report presents our initial attempt to build a spoken large language model (LLM) for Taiwanese Mandarin, specifically tailored to enable real-time, speech-to-speech interaction in multi-turn conversations. Our end-to-end model incorporates a decoder-only transformer architecture and aims to achieve seamless interaction while preserving the conversational flow, including full-duplex capabilities allowing simultaneous speaking and listening. The paper also details the training process, including data preparation with synthesized dialogues and adjustments for real-time interaction. We also developed a platform to evaluate conversational fluency and response coherence in multi-turn dialogues. We hope the release of the report can contribute to the future development of spoken LLMs in Taiwanese Mandarin. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_07111 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Building a Taiwanese Mandarin Spoken Language Model: A First Attempt Yang, Chih-Kai Fu, Yu-Kuan Li, Chen-An Lin, Yi-Cheng Lin, Yu-Xiang Chen, Wei-Chih Chung, Ho Lam Kuan, Chun-Yi Huang, Wei-Ping Lu, Ke-Han Lin, Tzu-Quan Wang, Hsiu-Hsuan Hu, En-Pei Hsu, Chan-Jan Tseng, Liang-Hsuan Chiu, I-Hsiang Sanga, Ulin Chen, Xuanjun Hsu, Po-chun Yang, Shu-wen Lee, Hung-yi Computation and Language Sound Audio and Speech Processing This technical report presents our initial attempt to build a spoken large language model (LLM) for Taiwanese Mandarin, specifically tailored to enable real-time, speech-to-speech interaction in multi-turn conversations. Our end-to-end model incorporates a decoder-only transformer architecture and aims to achieve seamless interaction while preserving the conversational flow, including full-duplex capabilities allowing simultaneous speaking and listening. The paper also details the training process, including data preparation with synthesized dialogues and adjustments for real-time interaction. We also developed a platform to evaluate conversational fluency and response coherence in multi-turn dialogues. We hope the release of the report can contribute to the future development of spoken LLMs in Taiwanese Mandarin. |
| title | Building a Taiwanese Mandarin Spoken Language Model: A First Attempt |
| topic | Computation and Language Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2411.07111 |