FinMCP-Bench: Benchmarking LLM Agents for Real-World Financial Tool Use under the Model Context Protocol
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912982949167104 |
|---|---|
| author | Zhu, Jie Tian, Yimin Li, Boyang Wu, Kehao Liang, Zhongzhi Li, Junhui Zhang, Xianyin Guo, Lifan Chen, Feng Liu, Yong Zhang, Chi |
| author_facet | Zhu, Jie Tian, Yimin Li, Boyang Wu, Kehao Liang, Zhongzhi Li, Junhui Zhang, Xianyin Guo, Lifan Chen, Feng Liu, Yong Zhang, Chi |
| contents | This paper introduces \textbf{FinMCP-Bench}, a novel benchmark for evaluating large language models (LLMs) in solving real-world financial problems through tool invocation of financial model context protocols. FinMCP-Bench contains 613 samples spanning 10 main scenarios and 33 sub-scenarios, featuring both real and synthetic user queries to ensure diversity and authenticity. It incorporates 65 real financial MCPs and three types of samples, single tool, multi-tool, and multi-turn, allowing evaluation of models across different levels of task complexity. Using this benchmark, we systematically assess a range of mainstream LLMs and propose metrics that explicitly measure tool invocation accuracy and reasoning capabilities. FinMCP-Bench provides a standardized, practical, and challenging testbed for advancing research on financial LLM agents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_24943 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | FinMCP-Bench: Benchmarking LLM Agents for Real-World Financial Tool Use under the Model Context Protocol Zhu, Jie Tian, Yimin Li, Boyang Wu, Kehao Liang, Zhongzhi Li, Junhui Zhang, Xianyin Guo, Lifan Chen, Feng Liu, Yong Zhang, Chi Artificial Intelligence Computation and Language This paper introduces \textbf{FinMCP-Bench}, a novel benchmark for evaluating large language models (LLMs) in solving real-world financial problems through tool invocation of financial model context protocols. FinMCP-Bench contains 613 samples spanning 10 main scenarios and 33 sub-scenarios, featuring both real and synthetic user queries to ensure diversity and authenticity. It incorporates 65 real financial MCPs and three types of samples, single tool, multi-tool, and multi-turn, allowing evaluation of models across different levels of task complexity. Using this benchmark, we systematically assess a range of mainstream LLMs and propose metrics that explicitly measure tool invocation accuracy and reasoning capabilities. FinMCP-Bench provides a standardized, practical, and challenging testbed for advancing research on financial LLM agents. |
| title | FinMCP-Bench: Benchmarking LLM Agents for Real-World Financial Tool Use under the Model Context Protocol |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2603.24943 |