Omni-R1: Do You Really Need Audio to Fine-Tune Your Audio LLM?
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918212959993856 |
|---|---|
| author | Rouditchenko, Andrew Bhati, Saurabhchand Araujo, Edson Thomas, Samuel Kuehne, Hilde Feris, Rogerio Glass, James |
| author_facet | Rouditchenko, Andrew Bhati, Saurabhchand Araujo, Edson Thomas, Samuel Kuehne, Hilde Feris, Rogerio Glass, James |
| contents | We propose Omni-R1 which fine-tunes a recent multi-modal LLM, Qwen2.5-Omni, on an audio question answering dataset with the reinforcement learning method GRPO. This leads to new State-of-the-Art performance on the recent MMAU and MMAR benchmarks. Omni-R1 achieves the highest accuracies on the sounds, music, speech, and overall average categories, both on the Test-mini and Test-full splits. To understand the performance improvement, we tested models both with and without audio and found that much of the performance improvement from GRPO could be attributed to better text-based reasoning. We also made a surprising discovery that fine-tuning without audio on a text-only dataset was effective at improving the audio-based performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_09439 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Omni-R1: Do You Really Need Audio to Fine-Tune Your Audio LLM? Rouditchenko, Andrew Bhati, Saurabhchand Araujo, Edson Thomas, Samuel Kuehne, Hilde Feris, Rogerio Glass, James Audio and Speech Processing Sound We propose Omni-R1 which fine-tunes a recent multi-modal LLM, Qwen2.5-Omni, on an audio question answering dataset with the reinforcement learning method GRPO. This leads to new State-of-the-Art performance on the recent MMAU and MMAR benchmarks. Omni-R1 achieves the highest accuracies on the sounds, music, speech, and overall average categories, both on the Test-mini and Test-full splits. To understand the performance improvement, we tested models both with and without audio and found that much of the performance improvement from GRPO could be attributed to better text-based reasoning. We also made a surprising discovery that fine-tuning without audio on a text-only dataset was effective at improving the audio-based performance. |
| title | Omni-R1: Do You Really Need Audio to Fine-Tune Your Audio LLM? |
| topic | Audio and Speech Processing Sound |
| url | https://arxiv.org/abs/2505.09439 |