Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.05373 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911423927418880 |
|---|---|
| author | Sun, Haoqin Lyu, Chenyang Zhao, Shiwan Ni, Xuanfan Kong, Xiangyu Wang, Longyue Luo, Weihua Qin, Yong |
| author_facet | Sun, Haoqin Lyu, Chenyang Zhao, Shiwan Ni, Xuanfan Kong, Xiangyu Wang, Longyue Luo, Weihua Qin, Yong |
| contents | Despite the growing success of Large Speech Language Models (LSLMs) in processing short-term acoustic signals, their extension to long-form audio understanding is severely bottlenecked. This limitation stems from the limited context length and the exorbitant memory footprints required for long-form inference. In this work, we propose Speech-XL, a new model that capitalizes on the intrinsic key-value (KV) sparsification capacity of Large Language Models (LLMs) to achieve high-ratio speech input compression. Specifically, we introduce a novel special token, the Speech Summarization Token (SST), for each speech interval to encapsulate the intra-interval speech information into its associated KV pairs. The SST module is trained via instruction fine-tuning, employing a curriculum learning strategy where the SST learns to compress information in a progressive manner--advancing from low-ratio (simple) to high-ratio (challenging) compression. Despite utilizing significantly less training data than other baselines, our model achieves highly competitive performance on major benchmarks, including LongSpeech and AUDIOMARATHON. By addressing the long-standing bottlenecks in long-form audio modeling, our approach offers a novel perspective on the condensation of extensive acoustic sequences. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_05373 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Speech-XL: Towards Long-Form Speech Understanding in Large Speech Language Models Sun, Haoqin Lyu, Chenyang Zhao, Shiwan Ni, Xuanfan Kong, Xiangyu Wang, Longyue Luo, Weihua Qin, Yong Sound Despite the growing success of Large Speech Language Models (LSLMs) in processing short-term acoustic signals, their extension to long-form audio understanding is severely bottlenecked. This limitation stems from the limited context length and the exorbitant memory footprints required for long-form inference. In this work, we propose Speech-XL, a new model that capitalizes on the intrinsic key-value (KV) sparsification capacity of Large Language Models (LLMs) to achieve high-ratio speech input compression. Specifically, we introduce a novel special token, the Speech Summarization Token (SST), for each speech interval to encapsulate the intra-interval speech information into its associated KV pairs. The SST module is trained via instruction fine-tuning, employing a curriculum learning strategy where the SST learns to compress information in a progressive manner--advancing from low-ratio (simple) to high-ratio (challenging) compression. Despite utilizing significantly less training data than other baselines, our model achieves highly competitive performance on major benchmarks, including LongSpeech and AUDIOMARATHON. By addressing the long-standing bottlenecks in long-form audio modeling, our approach offers a novel perspective on the condensation of extensive acoustic sequences. |
| title | Speech-XL: Towards Long-Form Speech Understanding in Large Speech Language Models |
| topic | Sound |
| url | https://arxiv.org/abs/2602.05373 |