Advanced Financial Reasoning at Scale: A Comprehensive Evaluation of Large Language Models on CFA Level III
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916959618072576 |
|---|---|
| author | Shetty, Pranam Upadhayaya, Abhisek Shah, Parth Mitesh Jagabathula, Srikanth Nayak, Shilpi Fee, Anna Joo |
| author_facet | Shetty, Pranam Upadhayaya, Abhisek Shah, Parth Mitesh Jagabathula, Srikanth Nayak, Shilpi Fee, Anna Joo |
| contents | As financial institutions increasingly adopt Large Language Models (LLMs), rigorous domain-specific evaluation becomes critical for responsible deployment. This paper presents a comprehensive benchmark evaluating 23 state-of-the-art LLMs on the Chartered Financial Analyst (CFA) Level III exam - the gold standard for advanced financial reasoning. We assess both multiple-choice questions (MCQs) and essay-style responses using multiple prompting strategies including Chain-of-Thought and Self-Discover. Our evaluation reveals that leading models demonstrate strong capabilities, with composite scores such as 79.1% (o4-mini) and 77.3% (Gemini 2.5 Flash) on CFA Level III. These results, achieved under a revised, stricter essay grading methodology, indicate significant progress in LLM capabilities for high-stakes financial applications. Our findings provide crucial guidance for practitioners on model selection and highlight remaining challenges in cost-effective deployment and the need for nuanced interpretation of performance against professional benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_02954 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Advanced Financial Reasoning at Scale: A Comprehensive Evaluation of Large Language Models on CFA Level III Shetty, Pranam Upadhayaya, Abhisek Shah, Parth Mitesh Jagabathula, Srikanth Nayak, Shilpi Fee, Anna Joo Computation and Language Artificial Intelligence As financial institutions increasingly adopt Large Language Models (LLMs), rigorous domain-specific evaluation becomes critical for responsible deployment. This paper presents a comprehensive benchmark evaluating 23 state-of-the-art LLMs on the Chartered Financial Analyst (CFA) Level III exam - the gold standard for advanced financial reasoning. We assess both multiple-choice questions (MCQs) and essay-style responses using multiple prompting strategies including Chain-of-Thought and Self-Discover. Our evaluation reveals that leading models demonstrate strong capabilities, with composite scores such as 79.1% (o4-mini) and 77.3% (Gemini 2.5 Flash) on CFA Level III. These results, achieved under a revised, stricter essay grading methodology, indicate significant progress in LLM capabilities for high-stakes financial applications. Our findings provide crucial guidance for practitioners on model selection and highlight remaining challenges in cost-effective deployment and the need for nuanced interpretation of performance against professional benchmarks. |
| title | Advanced Financial Reasoning at Scale: A Comprehensive Evaluation of Large Language Models on CFA Level III |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2507.02954 |