CORD: Bridging the Audio-Text Reasoning Gap via Weighted On-policy Cross-modal Distillation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914275372564480 |
|---|---|
| author | Hu, Jing Zhu, Danxiang Luo, Xianlong Zhang, Dan He, Shuwei Lei, Yishu Zheng, Haitao Feng, Shikun He, Jingzhou Sun, Yu Wu, Hua Wang, Haifeng |
| author_facet | Hu, Jing Zhu, Danxiang Luo, Xianlong Zhang, Dan He, Shuwei Lei, Yishu Zheng, Haitao Feng, Shikun He, Jingzhou Sun, Yu Wu, Hua Wang, Haifeng |
| contents | Large Audio Language Models (LALMs) have garnered significant research interest. Despite being built upon text-based large language models (LLMs), LALMs frequently exhibit a degradation in knowledge and reasoning capabilities. We hypothesize that this limitation stems from the failure of current training paradigms to effectively bridge the acoustic-semantic gap within the feature representation space. To address this challenge, we propose CORD, a unified alignment framework that performs online cross-modal self-distillation. Specifically, it aligns audio-conditioned reasoning with its text-conditioned counterpart within a unified model. Leveraging the text modality as an internal teacher, CORD performs multi-granularity alignment throughout the audio rollout process. At the token level, it employs on-policy reverse KL divergence with importance-aware weighting to prioritize early and semantically critical tokens. At the sequence level, CORD introduces a judge-based global reward to optimize complete reasoning trajectories via Group Relative Policy Optimization (GRPO). Empirical results across multiple benchmarks demonstrate that CORD consistently enhances audio-conditioned reasoning and substantially bridges the audio-text performance gap with only 80k synthetic training samples, validating the efficacy and data efficiency of our on-policy, multi-level cross-modal alignment approach. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_16547 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | CORD: Bridging the Audio-Text Reasoning Gap via Weighted On-policy Cross-modal Distillation Hu, Jing Zhu, Danxiang Luo, Xianlong Zhang, Dan He, Shuwei Lei, Yishu Zheng, Haitao Feng, Shikun He, Jingzhou Sun, Yu Wu, Hua Wang, Haifeng Sound Artificial Intelligence Audio and Speech Processing Large Audio Language Models (LALMs) have garnered significant research interest. Despite being built upon text-based large language models (LLMs), LALMs frequently exhibit a degradation in knowledge and reasoning capabilities. We hypothesize that this limitation stems from the failure of current training paradigms to effectively bridge the acoustic-semantic gap within the feature representation space. To address this challenge, we propose CORD, a unified alignment framework that performs online cross-modal self-distillation. Specifically, it aligns audio-conditioned reasoning with its text-conditioned counterpart within a unified model. Leveraging the text modality as an internal teacher, CORD performs multi-granularity alignment throughout the audio rollout process. At the token level, it employs on-policy reverse KL divergence with importance-aware weighting to prioritize early and semantically critical tokens. At the sequence level, CORD introduces a judge-based global reward to optimize complete reasoning trajectories via Group Relative Policy Optimization (GRPO). Empirical results across multiple benchmarks demonstrate that CORD consistently enhances audio-conditioned reasoning and substantially bridges the audio-text performance gap with only 80k synthetic training samples, validating the efficacy and data efficiency of our on-policy, multi-level cross-modal alignment approach. |
| title | CORD: Bridging the Audio-Text Reasoning Gap via Weighted On-policy Cross-modal Distillation |
| topic | Sound Artificial Intelligence Audio and Speech Processing |
| url | https://arxiv.org/abs/2601.16547 |