FinMMDocR: Benchmarking Financial Multimodal Reasoning with Scenario Awareness, Document Understanding, and Multi-Step Computation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918399526830080 |
|---|---|
| author | Tang, Zichen E, Haihong Li, Rongjin Liu, Jiacheng Jia, Linwei Hao, Zhuodi Yang, Zhongjun Li, Yuanze Tian, Haolin Hu, Xinyi Zhao, Peizhi Liu, Yuan Wang, Zhengyu Wang, Xianghe Huang, Yiling Lin, Xueyuan Bai, Ruofei Xie, Zijian Huang, Qian Cao, Ruining Gao, Haocheng |
| author_facet | Tang, Zichen E, Haihong Li, Rongjin Liu, Jiacheng Jia, Linwei Hao, Zhuodi Yang, Zhongjun Li, Yuanze Tian, Haolin Hu, Xinyi Zhao, Peizhi Liu, Yuan Wang, Zhengyu Wang, Xianghe Huang, Yiling Lin, Xueyuan Bai, Ruofei Xie, Zijian Huang, Qian Cao, Ruining Gao, Haocheng |
| contents | We introduce FinMMDocR, a novel bilingual multimodal benchmark for evaluating multimodal large language models (MLLMs) on real-world financial numerical reasoning. Compared to existing benchmarks, our work delivers three major advancements. (1) Scenario Awareness: 57.9% of 1,200 expert-annotated problems incorporate 12 types of implicit financial scenarios (e.g., Portfolio Management), challenging models to perform expert-level reasoning based on assumptions; (2) Document Understanding: 837 Chinese/English documents spanning 9 types (e.g., Company Research) average 50.8 pages with rich visual elements, significantly surpassing existing benchmarks in both breadth and depth of financial documents; (3) Multi-Step Computation: Problems demand 11-step reasoning on average (5.3 extraction + 5.7 calculation steps), with 65.0% requiring cross-page evidence (2.4 pages average). The best-performing MLLM achieves only 58.0% accuracy, and different retrieval-augmented generation (RAG) methods show significant performance variations on this task. We expect FinMMDocR to drive improvements in MLLMs and reasoning-enhanced methods on complex multimodal reasoning tasks in real-world scenarios. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_24903 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FinMMDocR: Benchmarking Financial Multimodal Reasoning with Scenario Awareness, Document Understanding, and Multi-Step Computation Tang, Zichen E, Haihong Li, Rongjin Liu, Jiacheng Jia, Linwei Hao, Zhuodi Yang, Zhongjun Li, Yuanze Tian, Haolin Hu, Xinyi Zhao, Peizhi Liu, Yuan Wang, Zhengyu Wang, Xianghe Huang, Yiling Lin, Xueyuan Bai, Ruofei Xie, Zijian Huang, Qian Cao, Ruining Gao, Haocheng Computer Vision and Pattern Recognition Computational Engineering, Finance, and Science We introduce FinMMDocR, a novel bilingual multimodal benchmark for evaluating multimodal large language models (MLLMs) on real-world financial numerical reasoning. Compared to existing benchmarks, our work delivers three major advancements. (1) Scenario Awareness: 57.9% of 1,200 expert-annotated problems incorporate 12 types of implicit financial scenarios (e.g., Portfolio Management), challenging models to perform expert-level reasoning based on assumptions; (2) Document Understanding: 837 Chinese/English documents spanning 9 types (e.g., Company Research) average 50.8 pages with rich visual elements, significantly surpassing existing benchmarks in both breadth and depth of financial documents; (3) Multi-Step Computation: Problems demand 11-step reasoning on average (5.3 extraction + 5.7 calculation steps), with 65.0% requiring cross-page evidence (2.4 pages average). The best-performing MLLM achieves only 58.0% accuracy, and different retrieval-augmented generation (RAG) methods show significant performance variations on this task. We expect FinMMDocR to drive improvements in MLLMs and reasoning-enhanced methods on complex multimodal reasoning tasks in real-world scenarios. |
| title | FinMMDocR: Benchmarking Financial Multimodal Reasoning with Scenario Awareness, Document Understanding, and Multi-Step Computation |
| topic | Computer Vision and Pattern Recognition Computational Engineering, Finance, and Science |
| url | https://arxiv.org/abs/2512.24903 |