MOSS-Audio Technical Report
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910280026423296 |
|---|---|
| author | Yang, Chen Yu, Chufan Chen, Hanfu Zhu, Jie Chen, Jingqi Chen, Ke Wang, Wenxuan Wang, Yang Jiang, Yaozhou Jiang, Yi Lin, Zhengyuan Chen, Ziqi Fei, Zhaoye Liu, Chenghao Zhan, Jun Yu, Kang Huang, Kexin Chen, Mingshu Cheng, Qinyuan Li, Ruixiao Li, Shimin Wang, Songlin Gao, Yang Zhang, Yiyang Qiu, Xipeng |
| author_facet | Yang, Chen Yu, Chufan Chen, Hanfu Zhu, Jie Chen, Jingqi Chen, Ke Wang, Wenxuan Wang, Yang Jiang, Yaozhou Jiang, Yi Lin, Zhengyuan Chen, Ziqi Fei, Zhaoye Liu, Chenghao Zhan, Jun Yu, Kang Huang, Kexin Chen, Mingshu Cheng, Qinyuan Li, Ruixiao Li, Shimin Wang, Songlin Gao, Yang Zhang, Yiyang Qiu, Xipeng |
| contents | MOSS-Audio is a unified audio-language model for speech, environmental sound, and music understanding, supporting audio captioning, time-aware question answering, timestamped transcription, and audio-grounded reasoning. MOSS-Audio couples a dedicated audio encoder with a modality adapter and a large language model: the encoder produces 12.5 Hz temporal representations, the adapter projects them into the decoder space, and the decoder generates autoregressive text outputs. Two design choices are central to the system: \textbf{DeepStack cross-layer feature injection}, which exposes the decoder to acoustic information from multiple encoder depths, and \textbf{time markers}, which provide explicit temporal cues by inserting timestamp markers into the audio-token stream. At the data level, we design an event-preserving audio annotation pipeline that segments raw audio at coherent event boundaries, applies branch-specific annotation to speech, music, and general audio, and merges the results into unified captions for pretraining. The intermediate branch-specific captions are further retained to support the construction of task-oriented SFT data. The model is pretrained on large-scale audio-language data, with time-aware objectives incorporated to support temporal grounding, and then undergoes multi-stage post-training to enhance instruction following and audio-grounded reasoning. We release 4B and 8B variants in both Instruct and Thinking configurations. MOSS-Audio achieves strong performance across general audio understanding, speech captioning, ASR, and timestamped ASR, positioning it as a promising understanding foundation for future voice agents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2606_01802 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | MOSS-Audio Technical Report Yang, Chen Yu, Chufan Chen, Hanfu Zhu, Jie Chen, Jingqi Chen, Ke Wang, Wenxuan Wang, Yang Jiang, Yaozhou Jiang, Yi Lin, Zhengyuan Chen, Ziqi Fei, Zhaoye Liu, Chenghao Zhan, Jun Yu, Kang Huang, Kexin Chen, Mingshu Cheng, Qinyuan Li, Ruixiao Li, Shimin Wang, Songlin Gao, Yang Zhang, Yiyang Qiu, Xipeng Sound Artificial Intelligence MOSS-Audio is a unified audio-language model for speech, environmental sound, and music understanding, supporting audio captioning, time-aware question answering, timestamped transcription, and audio-grounded reasoning. MOSS-Audio couples a dedicated audio encoder with a modality adapter and a large language model: the encoder produces 12.5 Hz temporal representations, the adapter projects them into the decoder space, and the decoder generates autoregressive text outputs. Two design choices are central to the system: \textbf{DeepStack cross-layer feature injection}, which exposes the decoder to acoustic information from multiple encoder depths, and \textbf{time markers}, which provide explicit temporal cues by inserting timestamp markers into the audio-token stream. At the data level, we design an event-preserving audio annotation pipeline that segments raw audio at coherent event boundaries, applies branch-specific annotation to speech, music, and general audio, and merges the results into unified captions for pretraining. The intermediate branch-specific captions are further retained to support the construction of task-oriented SFT data. The model is pretrained on large-scale audio-language data, with time-aware objectives incorporated to support temporal grounding, and then undergoes multi-stage post-training to enhance instruction following and audio-grounded reasoning. We release 4B and 8B variants in both Instruct and Thinking configurations. MOSS-Audio achieves strong performance across general audio understanding, speech captioning, ASR, and timestamped ASR, positioning it as a promising understanding foundation for future voice agents. |
| title | MOSS-Audio Technical Report |
| topic | Sound Artificial Intelligence |
| url | https://arxiv.org/abs/2606.01802 |