Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911081180430336 |
|---|---|
| author | Goel, Arushi Ghosh, Sreyan Kim, Jaehyeon Kumar, Sonal Kong, Zhifeng Lee, Sang-gil Yang, Chao-Han Huck Duraiswami, Ramani Manocha, Dinesh Valle, Rafael Catanzaro, Bryan |
| author_facet | Goel, Arushi Ghosh, Sreyan Kim, Jaehyeon Kumar, Sonal Kong, Zhifeng Lee, Sang-gil Yang, Chao-Han Huck Duraiswami, Ramani Manocha, Dinesh Valle, Rafael Catanzaro, Bryan |
| contents | We present Audio Flamingo 3 (AF3), a fully open state-of-the-art (SOTA) large audio-language model that advances reasoning and understanding across speech, sound, and music. AF3 introduces: (i) AF-Whisper, a unified audio encoder trained using a novel strategy for joint representation learning across all 3 modalities of speech, sound, and music; (ii) flexible, on-demand thinking, allowing the model to do chain-of-thought-type reasoning before answering; (iii) multi-turn, multi-audio chat; (iv) long audio understanding and reasoning (including speech) up to 10 minutes; and (v) voice-to-voice interaction. To enable these capabilities, we propose several large-scale training datasets curated using novel strategies, including AudioSkills-XL, LongAudio-XL, AF-Think, and AF-Chat, and train AF3 with a novel five-stage curriculum-based training strategy. Trained on only open-source audio data, AF3 achieves new SOTA results on over 20+ (long) audio understanding and reasoning benchmarks, surpassing both open-weight and closed-source models trained on much larger datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_08128 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models Goel, Arushi Ghosh, Sreyan Kim, Jaehyeon Kumar, Sonal Kong, Zhifeng Lee, Sang-gil Yang, Chao-Han Huck Duraiswami, Ramani Manocha, Dinesh Valle, Rafael Catanzaro, Bryan Sound Artificial Intelligence Computation and Language Audio and Speech Processing We present Audio Flamingo 3 (AF3), a fully open state-of-the-art (SOTA) large audio-language model that advances reasoning and understanding across speech, sound, and music. AF3 introduces: (i) AF-Whisper, a unified audio encoder trained using a novel strategy for joint representation learning across all 3 modalities of speech, sound, and music; (ii) flexible, on-demand thinking, allowing the model to do chain-of-thought-type reasoning before answering; (iii) multi-turn, multi-audio chat; (iv) long audio understanding and reasoning (including speech) up to 10 minutes; and (v) voice-to-voice interaction. To enable these capabilities, we propose several large-scale training datasets curated using novel strategies, including AudioSkills-XL, LongAudio-XL, AF-Think, and AF-Chat, and train AF3 with a novel five-stage curriculum-based training strategy. Trained on only open-source audio data, AF3 achieves new SOTA results on over 20+ (long) audio understanding and reasoning benchmarks, surpassing both open-weight and closed-source models trained on much larger datasets. |
| title | Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models |
| topic | Sound Artificial Intelligence Computation and Language Audio and Speech Processing |
| url | https://arxiv.org/abs/2507.08128 |