SECQUE: A Benchmark for Evaluating Real-World Financial Analysis Capabilities
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912539477016576 |
|---|---|
| author | Yoash, Noga Ben Brief, Meni Ovadia, Oded Shenderovitz, Gil Mishaeli, Moshik Lemberg, Rachel Sheetrit, Eitam |
| author_facet | Yoash, Noga Ben Brief, Meni Ovadia, Oded Shenderovitz, Gil Mishaeli, Moshik Lemberg, Rachel Sheetrit, Eitam |
| contents | We introduce SECQUE, a comprehensive benchmark for evaluating large language models (LLMs) in financial analysis tasks. SECQUE comprises 565 expert-written questions covering SEC filings analysis across four key categories: comparison analysis, ratio calculation, risk assessment, and financial insight generation. To assess model performance, we develop SECQUE-Judge, an evaluation mechanism leveraging multiple LLM-based judges, which demonstrates strong alignment with human evaluations. Additionally, we provide an extensive analysis of various models' performance on our benchmark. By making SECQUE publicly available, we aim to facilitate further research and advancements in financial AI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_04596 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SECQUE: A Benchmark for Evaluating Real-World Financial Analysis Capabilities Yoash, Noga Ben Brief, Meni Ovadia, Oded Shenderovitz, Gil Mishaeli, Moshik Lemberg, Rachel Sheetrit, Eitam Artificial Intelligence Computational Engineering, Finance, and Science Computation and Language We introduce SECQUE, a comprehensive benchmark for evaluating large language models (LLMs) in financial analysis tasks. SECQUE comprises 565 expert-written questions covering SEC filings analysis across four key categories: comparison analysis, ratio calculation, risk assessment, and financial insight generation. To assess model performance, we develop SECQUE-Judge, an evaluation mechanism leveraging multiple LLM-based judges, which demonstrates strong alignment with human evaluations. Additionally, we provide an extensive analysis of various models' performance on our benchmark. By making SECQUE publicly available, we aim to facilitate further research and advancements in financial AI. |
| title | SECQUE: A Benchmark for Evaluating Real-World Financial Analysis Capabilities |
| topic | Artificial Intelligence Computational Engineering, Finance, and Science Computation and Language |
| url | https://arxiv.org/abs/2504.04596 |