Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.18749 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917508477353984 |
|---|---|
| author | Zhou, Feiyan Wang, Luyuan Chen, Shoufa Wang, Zhe Liu, Zhiheng Cong, Yuren Zhang, Xiaohui Yang, Fanny Zeng, Belinda |
| author_facet | Zhou, Feiyan Wang, Luyuan Chen, Shoufa Wang, Zhe Liu, Zhiheng Cong, Yuren Zhang, Xiaohui Yang, Fanny Zeng, Belinda |
| contents | Modern audio generation predominantly relies on latent-space compression, introducing additional complexity and potential information loss. In this work, we challenge this paradigm with WavFlow, a framework that generates high-fidelity audio directly in raw waveform space without intermediate representations. To overcome the inherent difficulties of modeling high-dimensional and low-energy signals, we reshape audio into 2D token grids through waveform patchify and introduce amplitude lifting to align signal scales, enabling stable optimization via direct x-prediction in flow matching. To capture complex semantic alignment and temporal synchronization, we leverage an automated data pipeline to curate 5 million high-quality video-text-audio triplets, allowing the model to learn fine-grained acoustic patterns from scratch. Experimental results show that WavFlow achieves competitive performance on the video-to-audio benchmark VGGSound (FD_PaSST: 59.98, IS_PANNs: 17.40, DeSync: 0.44) and the text-to-audio benchmark AudioCaps (FD_PANNs: 10.63, IS_PANNs: 12.62), matching or exceeding the performance of established latent-based methods. Our work demonstrates that intermediate compression is not a prerequisite for high-quality synthesis, offering a simpler and more scalable alternative for multimodal audio generation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_18749 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | WavFlow: Audio Generation in Waveform Space Zhou, Feiyan Wang, Luyuan Chen, Shoufa Wang, Zhe Liu, Zhiheng Cong, Yuren Zhang, Xiaohui Yang, Fanny Zeng, Belinda Sound Computer Vision and Pattern Recognition Modern audio generation predominantly relies on latent-space compression, introducing additional complexity and potential information loss. In this work, we challenge this paradigm with WavFlow, a framework that generates high-fidelity audio directly in raw waveform space without intermediate representations. To overcome the inherent difficulties of modeling high-dimensional and low-energy signals, we reshape audio into 2D token grids through waveform patchify and introduce amplitude lifting to align signal scales, enabling stable optimization via direct x-prediction in flow matching. To capture complex semantic alignment and temporal synchronization, we leverage an automated data pipeline to curate 5 million high-quality video-text-audio triplets, allowing the model to learn fine-grained acoustic patterns from scratch. Experimental results show that WavFlow achieves competitive performance on the video-to-audio benchmark VGGSound (FD_PaSST: 59.98, IS_PANNs: 17.40, DeSync: 0.44) and the text-to-audio benchmark AudioCaps (FD_PANNs: 10.63, IS_PANNs: 12.62), matching or exceeding the performance of established latent-based methods. Our work demonstrates that intermediate compression is not a prerequisite for high-quality synthesis, offering a simpler and more scalable alternative for multimodal audio generation. |
| title | WavFlow: Audio Generation in Waveform Space |
| topic | Sound Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2605.18749 |