FinAudio: A Benchmark for Audio Large Language Models in Financial Applications
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911324637757440 |
|---|---|
| author | Cao, Yupeng Li, Haohang Yu, Yangyang Javaji, Shashidhar Reddy He, Yueru Huang, Jimin Xie, Qianqian Dimino, Fabrizio Liu, Xiao-yang Subbalakshmi, K. P. Qiu, Meikang Ananiadou, Sophia Nie, Jian-Yun |
| author_facet | Cao, Yupeng Li, Haohang Yu, Yangyang Javaji, Shashidhar Reddy He, Yueru Huang, Jimin Xie, Qianqian Dimino, Fabrizio Liu, Xiao-yang Subbalakshmi, K. P. Qiu, Meikang Ananiadou, Sophia Nie, Jian-Yun |
| contents | Audio Large Language Models (AudioLLMs) have received widespread attention and have significantly improved performance on audio tasks such as conversation, audio understanding, and automatic speech recognition (ASR). Despite these advancements, there is an absence of a benchmark for assessing AudioLLMs in financial scenarios, where audio data, such as earnings conference calls and CEO speeches, are crucial resources for financial analysis and investment decisions. In this paper, we introduce \textsc{FinAudio}, the first benchmark designed to evaluate the capacity of AudioLLMs in the financial domain. We first define three tasks based on the unique characteristics of the financial domain: 1) ASR for short financial audio, 2) ASR for long financial audio, and 3) summarization of long financial audio. Then, we curate two short and two long audio datasets, respectively, and develop a novel dataset for financial audio summarization, comprising the \textsc{FinAudio} benchmark. Then, we evaluate seven prevalent AudioLLMs on \textsc{FinAudio}. Our evaluation reveals the limitations of existing AudioLLMs in the financial domain and offers insights for improving AudioLLMs. All datasets and codes will be released. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_20990 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FinAudio: A Benchmark for Audio Large Language Models in Financial Applications Cao, Yupeng Li, Haohang Yu, Yangyang Javaji, Shashidhar Reddy He, Yueru Huang, Jimin Xie, Qianqian Dimino, Fabrizio Liu, Xiao-yang Subbalakshmi, K. P. Qiu, Meikang Ananiadou, Sophia Nie, Jian-Yun Computational Engineering, Finance, and Science Artificial Intelligence Multimedia Audio Large Language Models (AudioLLMs) have received widespread attention and have significantly improved performance on audio tasks such as conversation, audio understanding, and automatic speech recognition (ASR). Despite these advancements, there is an absence of a benchmark for assessing AudioLLMs in financial scenarios, where audio data, such as earnings conference calls and CEO speeches, are crucial resources for financial analysis and investment decisions. In this paper, we introduce \textsc{FinAudio}, the first benchmark designed to evaluate the capacity of AudioLLMs in the financial domain. We first define three tasks based on the unique characteristics of the financial domain: 1) ASR for short financial audio, 2) ASR for long financial audio, and 3) summarization of long financial audio. Then, we curate two short and two long audio datasets, respectively, and develop a novel dataset for financial audio summarization, comprising the \textsc{FinAudio} benchmark. Then, we evaluate seven prevalent AudioLLMs on \textsc{FinAudio}. Our evaluation reveals the limitations of existing AudioLLMs in the financial domain and offers insights for improving AudioLLMs. All datasets and codes will be released. |
| title | FinAudio: A Benchmark for Audio Large Language Models in Financial Applications |
| topic | Computational Engineering, Finance, and Science Artificial Intelligence Multimedia |
| url | https://arxiv.org/abs/2503.20990 |