Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.23765 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917362041618432 |
|---|---|
| author | Dinkel, Heinrich Sun, Xingwei Li, Gang Mei, Jiahao Niu, Yadong Liu, Jizhong Li, Xiyang Liao, Yifan Zhou, Jiahao Zhang, Junbo Luan, Jian |
| author_facet | Dinkel, Heinrich Sun, Xingwei Li, Gang Mei, Jiahao Niu, Yadong Liu, Jizhong Li, Xiyang Liao, Yifan Zhou, Jiahao Zhang, Junbo Luan, Jian |
| contents | This paper introduces DashengTokenizer, a continuous audio tokenizer engineered for joint use in both understanding and generation tasks. Unlike conventional approaches, which train acoustic tokenizers and subsequently integrate frozen semantic knowledge, our method inverts this paradigm: we leverage frozen semantic features and inject acoustic information. In linear evaluation across 22 diverse tasks, our method outperforms previous audio codec and audio encoder baselines by a significant margin while maintaining competitive audio reconstruction quality. Notably, we demonstrate that this acoustic injection improves performance for tasks such as speech emotion recognition, music understanding, and acoustic scene classification. We further evaluate the tokenizer's generative performance on text-to-audio (TTA), text-to-music (TTM), and speech enhancement (SE). Our approach surpasses standard variational autoencoder (VAE)-based methods on TTA and TTM tasks, while its effectiveness on SE underscores its capabilities as a general-purpose audio encoder. Finally, our results challenge the prevailing assumption that VAE-based architectures are a prerequisite for audio synthesis. Checkpoints are available at https://huggingface.co/mispeech/dashengtokenizer. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_23765 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | DashengTokenizer: One layer is enough for unified audio understanding and generation Dinkel, Heinrich Sun, Xingwei Li, Gang Mei, Jiahao Niu, Yadong Liu, Jizhong Li, Xiyang Liao, Yifan Zhou, Jiahao Zhang, Junbo Luan, Jian Sound Audio and Speech Processing This paper introduces DashengTokenizer, a continuous audio tokenizer engineered for joint use in both understanding and generation tasks. Unlike conventional approaches, which train acoustic tokenizers and subsequently integrate frozen semantic knowledge, our method inverts this paradigm: we leverage frozen semantic features and inject acoustic information. In linear evaluation across 22 diverse tasks, our method outperforms previous audio codec and audio encoder baselines by a significant margin while maintaining competitive audio reconstruction quality. Notably, we demonstrate that this acoustic injection improves performance for tasks such as speech emotion recognition, music understanding, and acoustic scene classification. We further evaluate the tokenizer's generative performance on text-to-audio (TTA), text-to-music (TTM), and speech enhancement (SE). Our approach surpasses standard variational autoencoder (VAE)-based methods on TTA and TTM tasks, while its effectiveness on SE underscores its capabilities as a general-purpose audio encoder. Finally, our results challenge the prevailing assumption that VAE-based architectures are a prerequisite for audio synthesis. Checkpoints are available at https://huggingface.co/mispeech/dashengtokenizer. |
| title | DashengTokenizer: One layer is enough for unified audio understanding and generation |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2602.23765 |