FinMTM: A Multi-Turn Multimodal Benchmark for Financial Reasoning and Agent Evaluation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912871756070912 |
|---|---|
| author | Zhang, Chenxi Gan, Ziliang Zhu, Liyun Pang, Youwei Zhang, Qing Zhang, Rongjunchen |
| author_facet | Zhang, Chenxi Gan, Ziliang Zhu, Liyun Pang, Youwei Zhang, Qing Zhang, Rongjunchen |
| contents | The financial domain poses substantial challenges for vision-language models (VLMs) due to specialized chart formats and knowledge-intensive reasoning requirements. However, existing financial benchmarks are largely single-turn and rely on a narrow set of question formats, limiting comprehensive evaluation in realistic application scenarios. To address this gap, we propose FinMTM, a multi-turn multimodal benchmark that expands diversity along both data and task dimensions. On the data side, we curate and annotate 11{,}133 bilingual (Chinese and English) financial QA pairs grounded in financial visuals, including candlestick charts, statistical plots, and report figures. On the task side, FinMTM covers single- and multiple-choice questions, multi-turn open-ended dialogues, and agent-based tasks. We further design task-specific evaluation protocols, including a set-overlap scoring rule for multiple-choice questions, a weighted combination of turn-level and session-level scores for multi-turn dialogues, and a composite metric that integrates planning quality with final outcomes for agent tasks. Extensive experimental evaluation of 22 VLMs reveal their limitations in fine-grained visual perception, long-context reasoning, and complex agent workflows. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_03130 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | FinMTM: A Multi-Turn Multimodal Benchmark for Financial Reasoning and Agent Evaluation Zhang, Chenxi Gan, Ziliang Zhu, Liyun Pang, Youwei Zhang, Qing Zhang, Rongjunchen Computer Vision and Pattern Recognition Computational Engineering, Finance, and Science The financial domain poses substantial challenges for vision-language models (VLMs) due to specialized chart formats and knowledge-intensive reasoning requirements. However, existing financial benchmarks are largely single-turn and rely on a narrow set of question formats, limiting comprehensive evaluation in realistic application scenarios. To address this gap, we propose FinMTM, a multi-turn multimodal benchmark that expands diversity along both data and task dimensions. On the data side, we curate and annotate 11{,}133 bilingual (Chinese and English) financial QA pairs grounded in financial visuals, including candlestick charts, statistical plots, and report figures. On the task side, FinMTM covers single- and multiple-choice questions, multi-turn open-ended dialogues, and agent-based tasks. We further design task-specific evaluation protocols, including a set-overlap scoring rule for multiple-choice questions, a weighted combination of turn-level and session-level scores for multi-turn dialogues, and a composite metric that integrates planning quality with final outcomes for agent tasks. Extensive experimental evaluation of 22 VLMs reveal their limitations in fine-grained visual perception, long-context reasoning, and complex agent workflows. |
| title | FinMTM: A Multi-Turn Multimodal Benchmark for Financial Reasoning and Agent Evaluation |
| topic | Computer Vision and Pattern Recognition Computational Engineering, Finance, and Science |
| url | https://arxiv.org/abs/2602.03130 |