Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.22273 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917294861451264 |
|---|---|
| author | Zhang, Xiyuan Wu, Huihang Guo, Jiayu Zhang, Zhenlin Zhang, Yiwei Huo, Liangyu Ma, Xiaoxiao Wan, Jiansong Jiao, Xuewei Jing, Yi Xie, Jian |
| author_facet | Zhang, Xiyuan Wu, Huihang Guo, Jiayu Zhang, Zhenlin Zhang, Yiwei Huo, Liangyu Ma, Xiaoxiao Wan, Jiansong Jiao, Xuewei Jing, Yi Xie, Jian |
| contents | We introduce FIRE, a comprehensive benchmark designed to evaluate both the theoretical financial knowledge of LLMs and their ability to handle practical business scenarios. For theoretical assessment, we curate a diverse set of examination questions drawn from widely recognized financial qualification exams, enabling evaluation of LLMs deep understanding and application of financial knowledge. In addition, to assess the practical value of LLMs in real-world financial tasks, we propose a systematic evaluation matrix that categorizes complex financial domains and ensures coverage of essential subdomains and business activities. Based on this evaluation matrix, we collect 3,000 financial scenario questions, consisting of closed-form decision questions with reference answers and open-ended questions evaluated by predefined rubrics. We conduct comprehensive evaluations of state-of-the-art LLMs on the FIRE benchmark, including XuanYuan 4.0, our latest financial-domain model, as a strong in-domain baseline. These results enable a systematic analysis of the capability boundaries of current LLMs in financial applications. We publicly release the benchmark questions and evaluation code to facilitate future research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_22273 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | FIRE: A Comprehensive Benchmark for Financial Intelligence and Reasoning Evaluation Zhang, Xiyuan Wu, Huihang Guo, Jiayu Zhang, Zhenlin Zhang, Yiwei Huo, Liangyu Ma, Xiaoxiao Wan, Jiansong Jiao, Xuewei Jing, Yi Xie, Jian Artificial Intelligence Machine Learning We introduce FIRE, a comprehensive benchmark designed to evaluate both the theoretical financial knowledge of LLMs and their ability to handle practical business scenarios. For theoretical assessment, we curate a diverse set of examination questions drawn from widely recognized financial qualification exams, enabling evaluation of LLMs deep understanding and application of financial knowledge. In addition, to assess the practical value of LLMs in real-world financial tasks, we propose a systematic evaluation matrix that categorizes complex financial domains and ensures coverage of essential subdomains and business activities. Based on this evaluation matrix, we collect 3,000 financial scenario questions, consisting of closed-form decision questions with reference answers and open-ended questions evaluated by predefined rubrics. We conduct comprehensive evaluations of state-of-the-art LLMs on the FIRE benchmark, including XuanYuan 4.0, our latest financial-domain model, as a strong in-domain baseline. These results enable a systematic analysis of the capability boundaries of current LLMs in financial applications. We publicly release the benchmark questions and evaluation code to facilitate future research. |
| title | FIRE: A Comprehensive Benchmark for Financial Intelligence and Reasoning Evaluation |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2602.22273 |