Competitive Audio-Language Models with Data-Efficient Single-Stage Training on Public Data
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918299227389952 |
|---|---|
| author | Kumar, Gokul Karthik Saraf, Rishabh Lepauloux, Ludovick Muneer, Abdul Mokeddem, Billel Hacid, Hakim |
| author_facet | Kumar, Gokul Karthik Saraf, Rishabh Lepauloux, Ludovick Muneer, Abdul Mokeddem, Billel Hacid, Hakim |
| contents | Large language models (LLMs) have transformed NLP, yet their integration with audio remains underexplored despite audio's centrality to human communication. We introduce Falcon3-Audio, a family of Audio-Language Models (ALMs) built on instruction-tuned LLMs and Whisper encoders. Using a remarkably small amount of public audio data, less than 30K hours (5K unique), Falcon3-Audio-7B matches the best reported performance among open-weight models on the MMAU benchmark, with a score of 64.14, matching R1-AQA, while distinguishing itself through superior data and parameter efficiency, single-stage training, and transparency. Notably, our smallest 1B model remains competitive with larger open models ranging from 2B to 13B parameters. Through extensive ablations, we find that common complexities such as curriculum learning, multiple audio encoders, and intricate cross-attention connectors are not required for strong performance, even compared to models trained on over 500K hours of data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_07526 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Competitive Audio-Language Models with Data-Efficient Single-Stage Training on Public Data Kumar, Gokul Karthik Saraf, Rishabh Lepauloux, Ludovick Muneer, Abdul Mokeddem, Billel Hacid, Hakim Sound Artificial Intelligence Computation and Language Machine Learning Large language models (LLMs) have transformed NLP, yet their integration with audio remains underexplored despite audio's centrality to human communication. We introduce Falcon3-Audio, a family of Audio-Language Models (ALMs) built on instruction-tuned LLMs and Whisper encoders. Using a remarkably small amount of public audio data, less than 30K hours (5K unique), Falcon3-Audio-7B matches the best reported performance among open-weight models on the MMAU benchmark, with a score of 64.14, matching R1-AQA, while distinguishing itself through superior data and parameter efficiency, single-stage training, and transparency. Notably, our smallest 1B model remains competitive with larger open models ranging from 2B to 13B parameters. Through extensive ablations, we find that common complexities such as curriculum learning, multiple audio encoders, and intricate cross-attention connectors are not required for strong performance, even compared to models trained on over 500K hours of data. |
| title | Competitive Audio-Language Models with Data-Efficient Single-Stage Training on Public Data |
| topic | Sound Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2509.07526 |