Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.01903 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908692781203456 |
|---|---|
| author | Li, Jingyi Zhao, Zhiyuan Zhang, Zhisheng Liu, Yunfei Lin, Lijian Zhu, Ye Wu, Jiahao Kong, Qiuqiang Li, Yu |
| author_facet | Li, Jingyi Zhao, Zhiyuan Zhang, Zhisheng Liu, Yunfei Lin, Lijian Zhu, Ye Wu, Jiahao Kong, Qiuqiang Li, Yu |
| contents | Large Audio Language Models (LALMs) have emerged with strong performance across diverse audio understanding tasks and can be further enhanced by neural audio codecs. Transitioning from multi-layer residual vector quantizers to a single-layer quantizer has been shown to facilitate more efficient downstream language models decoding. However, the ability of a single codebook to capture fine-grained acoustic details remains limited, as the frequency-variant nature of 1D tokenizers leads to redundancy. To address this issue, we propose MelTok, a two-dimensional (2D) tokenizer that effectively compresses acoustic details of 44.1 KHz audio into a single codebook. The tokenizer encodes audio into a more compact representation than one-dimensional tokenizers. Furthermore, to recover audio from mel-spectrogram tokens, we propose a token-based vocoder. Both objective and subjective evaluations demonstrate that MelTok achieves quality comparable to multi-codebook codecs and outperforms existing state-of-the-art neural codecs with a single codebook on high-fidelity audio reconstruction. By preserving acoustic details, MelTok offers a strong representation for downstream understanding tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_01903 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MelTok: 2D Tokenization for Single-Codebook Audio Compression Li, Jingyi Zhao, Zhiyuan Zhang, Zhisheng Liu, Yunfei Lin, Lijian Zhu, Ye Wu, Jiahao Kong, Qiuqiang Li, Yu Sound Audio and Speech Processing Large Audio Language Models (LALMs) have emerged with strong performance across diverse audio understanding tasks and can be further enhanced by neural audio codecs. Transitioning from multi-layer residual vector quantizers to a single-layer quantizer has been shown to facilitate more efficient downstream language models decoding. However, the ability of a single codebook to capture fine-grained acoustic details remains limited, as the frequency-variant nature of 1D tokenizers leads to redundancy. To address this issue, we propose MelTok, a two-dimensional (2D) tokenizer that effectively compresses acoustic details of 44.1 KHz audio into a single codebook. The tokenizer encodes audio into a more compact representation than one-dimensional tokenizers. Furthermore, to recover audio from mel-spectrogram tokens, we propose a token-based vocoder. Both objective and subjective evaluations demonstrate that MelTok achieves quality comparable to multi-codebook codecs and outperforms existing state-of-the-art neural codecs with a single codebook on high-fidelity audio reconstruction. By preserving acoustic details, MelTok offers a strong representation for downstream understanding tasks. |
| title | MelTok: 2D Tokenization for Single-Codebook Audio Compression |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2510.01903 |