Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.20224 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917376550764544 |
|---|---|
| author | Lin, Rui Wu, Zhiyue Le, Jiahe Wang, Kangdi Chen, Weixiong Dai, Junyu Jiang, Tao |
| author_facet | Lin, Rui Wu, Zhiyue Le, Jiahe Wang, Kangdi Chen, Weixiong Dai, Junyu Jiang, Tao |
| contents | Audio tokenization bridges continuous waveforms and multi-track music language models. In dual-track modeling, tokens should preserve three properties at once: high-fidelity reconstruction, strong predictability under a language model, and cross-track correspondence. We introduce DuoTok, a source-aware dual-track tokenizer that addresses this trade-off through staged disentanglement. DuoTok first pretrains a semantic encoder, then regularizes it with multi-task supervision, freezes the encoder, and applies hard dual-codebook routing while keeping auxiliary objectives on quantized codes. A diffusion decoder reconstructs high-frequency details, allowing tokens to focus on structured information for sequence modeling. On standard benchmarks, DuoTok achieves a favorable predictability-fidelity trade-off, reaching the lowest cnBPT while maintaining competitive reconstruction at 0.75 kbps. Under a held-constant dual-track language modeling protocol, enBPT also improves, indicating gains beyond codebook size effects. Controlled diagnostics show larger predictability costs under cross-track corruption and larger gains from longer context, suggesting that models trained on DuoTok tokens use cross-track structure and non-local history. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_20224 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DuoTok: Source-Aware Dual-Track Tokenization for Multi-Track Music Language Modeling Lin, Rui Wu, Zhiyue Le, Jiahe Wang, Kangdi Chen, Weixiong Dai, Junyu Jiang, Tao Sound Artificial Intelligence Audio tokenization bridges continuous waveforms and multi-track music language models. In dual-track modeling, tokens should preserve three properties at once: high-fidelity reconstruction, strong predictability under a language model, and cross-track correspondence. We introduce DuoTok, a source-aware dual-track tokenizer that addresses this trade-off through staged disentanglement. DuoTok first pretrains a semantic encoder, then regularizes it with multi-task supervision, freezes the encoder, and applies hard dual-codebook routing while keeping auxiliary objectives on quantized codes. A diffusion decoder reconstructs high-frequency details, allowing tokens to focus on structured information for sequence modeling. On standard benchmarks, DuoTok achieves a favorable predictability-fidelity trade-off, reaching the lowest cnBPT while maintaining competitive reconstruction at 0.75 kbps. Under a held-constant dual-track language modeling protocol, enBPT also improves, indicating gains beyond codebook size effects. Controlled diagnostics show larger predictability costs under cross-track corruption and larger gains from longer context, suggesting that models trained on DuoTok tokens use cross-track structure and non-local history. |
| title | DuoTok: Source-Aware Dual-Track Tokenization for Multi-Track Music Language Modeling |
| topic | Sound Artificial Intelligence |
| url | https://arxiv.org/abs/2511.20224 |