When Audio Generators Become Good Listeners: Generative Features for Understanding Tasks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915521818001408 |
|---|---|
| author | Xie, Zeyu Li, Chenxing Xu, Xuenan Wu, Mengyue Wang, Wenfu Fu, Ruibo Yu, Meng Yu, Dong Zou, Yuexian |
| author_facet | Xie, Zeyu Li, Chenxing Xu, Xuenan Wu, Mengyue Wang, Wenfu Fu, Ruibo Yu, Meng Yu, Dong Zou, Yuexian |
| contents | This work pioneers the utilization of generative features in enhancing audio understanding. Unlike conventional discriminative features that directly optimize posterior and thus emphasize semantic abstraction while losing fine grained details, audio generation models inherently encode both spatiotemporal perception (capturing local acoustic texture across time and frequency) and semantic prior (knowing what to generate). It motivates us to explore the bridge of these complementary strengths. We provide a systematic investigation of their differences and complementary relationships, and ultimately propose an effective fusion strategy. Experiments across multiple tasks, including sound event classification, tagging, and particularly the fine grained task of audio captioning, demonstrate consistent performance gains. Beyond empirical improvements, this work more importantly introduces a new perspective on audio representation learning, highlighting that generative discriminative complementarity can provide both detailed perception and semantic awareness for audio understanding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_24635 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | When Audio Generators Become Good Listeners: Generative Features for Understanding Tasks Xie, Zeyu Li, Chenxing Xu, Xuenan Wu, Mengyue Wang, Wenfu Fu, Ruibo Yu, Meng Yu, Dong Zou, Yuexian Sound 68Txx I.2 This work pioneers the utilization of generative features in enhancing audio understanding. Unlike conventional discriminative features that directly optimize posterior and thus emphasize semantic abstraction while losing fine grained details, audio generation models inherently encode both spatiotemporal perception (capturing local acoustic texture across time and frequency) and semantic prior (knowing what to generate). It motivates us to explore the bridge of these complementary strengths. We provide a systematic investigation of their differences and complementary relationships, and ultimately propose an effective fusion strategy. Experiments across multiple tasks, including sound event classification, tagging, and particularly the fine grained task of audio captioning, demonstrate consistent performance gains. Beyond empirical improvements, this work more importantly introduces a new perspective on audio representation learning, highlighting that generative discriminative complementarity can provide both detailed perception and semantic awareness for audio understanding. |
| title | When Audio Generators Become Good Listeners: Generative Features for Understanding Tasks |
| topic | Sound 68Txx I.2 |
| url | https://arxiv.org/abs/2509.24635 |