Evaluation and Benchmarking Suite for Financial Large Language Models and Agents
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917286721355776 |
|---|---|
| author | Lin, Shengyuan He, Kaiwen Patel, Jaisal Zhang, Qinchuan Ding, Chris Tang, James Wang, Keyi Cao, Yupeng Wang, Yan Xiao, Kairong Caldeira, Vincent White, Matt Yanglet, Xiao-Yang Liu |
| author_facet | Lin, Shengyuan He, Kaiwen Patel, Jaisal Zhang, Qinchuan Ding, Chris Tang, James Wang, Keyi Cao, Yupeng Wang, Yan Xiao, Kairong Caldeira, Vincent White, Matt Yanglet, Xiao-Yang Liu |
| contents | Over the past three years, the financial services industry has witnessed Large Language Models (LLMs) and agents transitioning from the exploration stage to readiness and governance stages. Financial large language models (FinLLMs), such as open FinGPT and proprietary BloombergGPT , have great potential in financial applications, including retrieving real-time data, tutoring, analyzing sentiment of social media, analyzing SEC filings, and agentic trading. However, general-purpose LLMs and agents lack financial expertise and often struggle to handle complex financial reasoning. This paper presents an evaluation and benchmarking suite that covers the lifecycle of FinLLMs and FinAgents. This suite led by SecureFinAI Lab includes an evaluation pipeline and a governance framework collaborating with Linux Foundation and PyTorch Foundation, a FinLLM Leaderboard with HuggingFace, an AgentOps framework with Red Hat, and a documentation website with Rensselear Center of Open Source. Our collaborative development evolves through three stages: FinLLM Exploration (2023), FinLLM Readiness (2024), and FinAI Governance (2025). The proposed suite serves as an open platform that enables researchers and practitioners to perform both quantitative and qualitative analysis of different FinLLMs and FinAgents, fostering a more robust and reliable FinAI ecosystem. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_19073 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Evaluation and Benchmarking Suite for Financial Large Language Models and Agents Lin, Shengyuan He, Kaiwen Patel, Jaisal Zhang, Qinchuan Ding, Chris Tang, James Wang, Keyi Cao, Yupeng Wang, Yan Xiao, Kairong Caldeira, Vincent White, Matt Yanglet, Xiao-Yang Liu Computational Engineering, Finance, and Science Over the past three years, the financial services industry has witnessed Large Language Models (LLMs) and agents transitioning from the exploration stage to readiness and governance stages. Financial large language models (FinLLMs), such as open FinGPT and proprietary BloombergGPT , have great potential in financial applications, including retrieving real-time data, tutoring, analyzing sentiment of social media, analyzing SEC filings, and agentic trading. However, general-purpose LLMs and agents lack financial expertise and often struggle to handle complex financial reasoning. This paper presents an evaluation and benchmarking suite that covers the lifecycle of FinLLMs and FinAgents. This suite led by SecureFinAI Lab includes an evaluation pipeline and a governance framework collaborating with Linux Foundation and PyTorch Foundation, a FinLLM Leaderboard with HuggingFace, an AgentOps framework with Red Hat, and a documentation website with Rensselear Center of Open Source. Our collaborative development evolves through three stages: FinLLM Exploration (2023), FinLLM Readiness (2024), and FinAI Governance (2025). The proposed suite serves as an open platform that enables researchers and practitioners to perform both quantitative and qualitative analysis of different FinLLMs and FinAgents, fostering a more robust and reliable FinAI ecosystem. |
| title | Evaluation and Benchmarking Suite for Financial Large Language Models and Agents |
| topic | Computational Engineering, Finance, and Science |
| url | https://arxiv.org/abs/2602.19073 |