VisFinEval: A Scenario-Driven Chinese Multimodal Benchmark for Holistic Financial Understanding
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909735770390528 |
|---|---|
| author | Liu, Zhaowei Guo, Xin Xia, Haotian Zeng, Lingfeng Lou, Fangqi Niu, Jinyi Li, Mengping Qi, Qi Li, Jiahuan Zhang, Wei Wang, Yinglong Cai, Weige Shen, Weining Zhang, Liwen |
| author_facet | Liu, Zhaowei Guo, Xin Xia, Haotian Zeng, Lingfeng Lou, Fangqi Niu, Jinyi Li, Mengping Qi, Qi Li, Jiahuan Zhang, Wei Wang, Yinglong Cai, Weige Shen, Weining Zhang, Liwen |
| contents | Multimodal large language models (MLLMs) hold great promise for automating complex financial analysis. To comprehensively evaluate their capabilities, we introduce VisFinEval, the first large-scale Chinese benchmark that spans the full front-middle-back office lifecycle of financial tasks. VisFinEval comprises 15,848 annotated question-answer pairs drawn from eight common financial image modalities (e.g., K-line charts, financial statements, official seals), organized into three hierarchical scenario depths: Financial Knowledge & Data Analysis, Financial Analysis & Decision Support, and Financial Risk Control & Asset Optimization. We evaluate 21 state-of-the-art MLLMs in a zero-shot setting. The top model, Qwen-VL-max, achieves an overall accuracy of 76.3%, outperforming non-expert humans but trailing financial experts by over 14 percentage points. Our error analysis uncovers six recurring failure modes-including cross-modal misalignment, hallucinations, and lapses in business-process reasoning-that highlight critical avenues for future research. VisFinEval aims to accelerate the development of robust, domain-tailored MLLMs capable of seamlessly integrating textual and visual financial information. The data and the code are available at https://github.com/SUFE-AIFLM-Lab/VisFinEval. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_09641 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VisFinEval: A Scenario-Driven Chinese Multimodal Benchmark for Holistic Financial Understanding Liu, Zhaowei Guo, Xin Xia, Haotian Zeng, Lingfeng Lou, Fangqi Niu, Jinyi Li, Mengping Qi, Qi Li, Jiahuan Zhang, Wei Wang, Yinglong Cai, Weige Shen, Weining Zhang, Liwen Computational Engineering, Finance, and Science Multimodal large language models (MLLMs) hold great promise for automating complex financial analysis. To comprehensively evaluate their capabilities, we introduce VisFinEval, the first large-scale Chinese benchmark that spans the full front-middle-back office lifecycle of financial tasks. VisFinEval comprises 15,848 annotated question-answer pairs drawn from eight common financial image modalities (e.g., K-line charts, financial statements, official seals), organized into three hierarchical scenario depths: Financial Knowledge & Data Analysis, Financial Analysis & Decision Support, and Financial Risk Control & Asset Optimization. We evaluate 21 state-of-the-art MLLMs in a zero-shot setting. The top model, Qwen-VL-max, achieves an overall accuracy of 76.3%, outperforming non-expert humans but trailing financial experts by over 14 percentage points. Our error analysis uncovers six recurring failure modes-including cross-modal misalignment, hallucinations, and lapses in business-process reasoning-that highlight critical avenues for future research. VisFinEval aims to accelerate the development of robust, domain-tailored MLLMs capable of seamlessly integrating textual and visual financial information. The data and the code are available at https://github.com/SUFE-AIFLM-Lab/VisFinEval. |
| title | VisFinEval: A Scenario-Driven Chinese Multimodal Benchmark for Holistic Financial Understanding |
| topic | Computational Engineering, Finance, and Science |
| url | https://arxiv.org/abs/2508.09641 |