Discrete Token Modeling for Multi-Stem Music Source Separation with Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913040385966080 |
|---|---|
| author | Lyu, Pengbo Zhao, Xiangyu Liu, Chengwei Yan, Haoyin Liang, Xiaotao Wang, Hongyu Xue, Shaofei |
| author_facet | Lyu, Pengbo Zhao, Xiangyu Liu, Chengwei Yan, Haoyin Liang, Xiaotao Wang, Hongyu Xue, Shaofei |
| contents | We propose a generative framework for multi-track music source separation (MSS) that reformulates the task as conditional discrete token generation. Unlike conventional approaches that directly estimate continuous signals in the time or frequency domain, our method combines a Conformer-based conditional encoder, a dual-path neural audio codec (HCodec), and a decoder-only language model to autoregressively generate audio tokens for four target tracks. The generated tokens are decoded back to waveforms through the codec decoder. Evaluation on the MUSDB18-HQ benchmark shows that our generative approach achieves perceptual quality approaching state-of-the-art discriminative methods, while attaining the highest NISQA score on the vocals track. Ablation studies confirm the effectiveness of the learnable Conformer encoder and the benefit of sequential cross-track generation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_09371 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Discrete Token Modeling for Multi-Stem Music Source Separation with Language Models Lyu, Pengbo Zhao, Xiangyu Liu, Chengwei Yan, Haoyin Liang, Xiaotao Wang, Hongyu Xue, Shaofei Audio and Speech Processing We propose a generative framework for multi-track music source separation (MSS) that reformulates the task as conditional discrete token generation. Unlike conventional approaches that directly estimate continuous signals in the time or frequency domain, our method combines a Conformer-based conditional encoder, a dual-path neural audio codec (HCodec), and a decoder-only language model to autoregressively generate audio tokens for four target tracks. The generated tokens are decoded back to waveforms through the codec decoder. Evaluation on the MUSDB18-HQ benchmark shows that our generative approach achieves perceptual quality approaching state-of-the-art discriminative methods, while attaining the highest NISQA score on the vocals track. Ablation studies confirm the effectiveness of the learnable Conformer encoder and the benefit of sequential cross-track generation. |
| title | Discrete Token Modeling for Multi-Stem Music Source Separation with Language Models |
| topic | Audio and Speech Processing |
| url | https://arxiv.org/abs/2604.09371 |