Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.22243 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918148820697088 |
|---|---|
| author | Ge, Yuan Chen, Saihan Xiao, Jingqi Liu, Xiaoqian Xiao, Tong Xiang, Yan Yu, Zhengtao Zhu, Jingbo |
| author_facet | Ge, Yuan Chen, Saihan Xiao, Jingqi Liu, Xiaoqian Xiao, Tong Xiang, Yan Yu, Zhengtao Zhu, Jingbo |
| contents | Full-Duplex Speech-to-Speech Large Language Models (LLMs) are foundational to natural human-computer interaction, enabling real-time spoken dialogue systems. However, benchmarking and modeling these models remains a fundamental challenge. We introduce FLEXI, the first benchmark for full-duplex LLM-human spoken interaction that explicitly incorporates model interruption in emergency scenarios. FLEXI systematically evaluates the latency, quality, and conversational effectiveness of real-time dialogue through six diverse human-LLM interaction scenarios, revealing significant gaps between open source and commercial models in emergency awareness, turn terminating, and interaction latency. Finally, we suggest that next token-pair prediction offers a promising path toward achieving truly seamless and human-like full-duplex interaction. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_22243 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FLEXI: Benchmarking Full-duplex Human-LLM Speech Interaction Ge, Yuan Chen, Saihan Xiao, Jingqi Liu, Xiaoqian Xiao, Tong Xiang, Yan Yu, Zhengtao Zhu, Jingbo Computation and Language Full-Duplex Speech-to-Speech Large Language Models (LLMs) are foundational to natural human-computer interaction, enabling real-time spoken dialogue systems. However, benchmarking and modeling these models remains a fundamental challenge. We introduce FLEXI, the first benchmark for full-duplex LLM-human spoken interaction that explicitly incorporates model interruption in emergency scenarios. FLEXI systematically evaluates the latency, quality, and conversational effectiveness of real-time dialogue through six diverse human-LLM interaction scenarios, revealing significant gaps between open source and commercial models in emergency awareness, turn terminating, and interaction latency. Finally, we suggest that next token-pair prediction offers a promising path toward achieving truly seamless and human-like full-duplex interaction. |
| title | FLEXI: Benchmarking Full-duplex Human-LLM Speech Interaction |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2509.22243 |