SECQUE: A Benchmark for Evaluating Real-World Financial Analysis Capabilities

Fuente: arXiv
Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Yoash, Noga Ben, Brief, Meni, Ovadia, Oded, Shenderovitz, Gil, Mishaeli, Moshik, Lemberg, Rachel, Sheetrit, Eitam
Format: Preprint
Veröffentlicht: 2025
Schlagworte:
Online-Zugang:
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
_version_ 1866912539477016576
author Yoash, Noga Ben
Brief, Meni
Ovadia, Oded
Shenderovitz, Gil
Mishaeli, Moshik
Lemberg, Rachel
Sheetrit, Eitam
author_facet Yoash, Noga Ben
Brief, Meni
Ovadia, Oded
Shenderovitz, Gil
Mishaeli, Moshik
Lemberg, Rachel
Sheetrit, Eitam
contents We introduce SECQUE, a comprehensive benchmark for evaluating large language models (LLMs) in financial analysis tasks. SECQUE comprises 565 expert-written questions covering SEC filings analysis across four key categories: comparison analysis, ratio calculation, risk assessment, and financial insight generation. To assess model performance, we develop SECQUE-Judge, an evaluation mechanism leveraging multiple LLM-based judges, which demonstrates strong alignment with human evaluations. Additionally, we provide an extensive analysis of various models' performance on our benchmark. By making SECQUE publicly available, we aim to facilitate further research and advancements in financial AI.
format Preprint
id arxiv_https___arxiv_org_abs_2504_04596
institution arXiv
publishDate 2025
record_format arxiv
spellingShingle SECQUE: A Benchmark for Evaluating Real-World Financial Analysis Capabilities
Yoash, Noga Ben
Brief, Meni
Ovadia, Oded
Shenderovitz, Gil
Mishaeli, Moshik
Lemberg, Rachel
Sheetrit, Eitam
Artificial Intelligence
Computational Engineering, Finance, and Science
Computation and Language
We introduce SECQUE, a comprehensive benchmark for evaluating large language models (LLMs) in financial analysis tasks. SECQUE comprises 565 expert-written questions covering SEC filings analysis across four key categories: comparison analysis, ratio calculation, risk assessment, and financial insight generation. To assess model performance, we develop SECQUE-Judge, an evaluation mechanism leveraging multiple LLM-based judges, which demonstrates strong alignment with human evaluations. Additionally, we provide an extensive analysis of various models' performance on our benchmark. By making SECQUE publicly available, we aim to facilitate further research and advancements in financial AI.
title SECQUE: A Benchmark for Evaluating Real-World Financial Analysis Capabilities
topic Artificial Intelligence
Computational Engineering, Finance, and Science
Computation and Language
url https://arxiv.org/abs/2504.04596