FinTagging: Benchmarking LLMs for Extracting and Structuring Financial Information
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910226981060608 |
|---|---|
| author | Wang, Yan Qian, Lingfei Peng, Xueqing Ren, Yang Wang, Keyi Han, Yi Feng, Dongji Mo, Fengran Lin, Shengyuan Zhang, Qinchuan He, Kaiwen Luo, Chenri Chen, Jianxing Wu, Junwei Xu, Chen Xu, Ziyang Huang, Jimin Xiong, Guojun Liu, Xiao-Yang Xie, Qianqian Nie, Jian-Yun |
| author_facet | Wang, Yan Qian, Lingfei Peng, Xueqing Ren, Yang Wang, Keyi Han, Yi Feng, Dongji Mo, Fengran Lin, Shengyuan Zhang, Qinchuan He, Kaiwen Luo, Chenri Chen, Jianxing Wu, Junwei Xu, Chen Xu, Ziyang Huang, Jimin Xiong, Guojun Liu, Xiao-Yang Xie, Qianqian Nie, Jian-Yun |
| contents | Accurate interpretation of numerical data in financial reports is critical for markets and regulators. Although XBRL (eXtensible Business Reporting Language) provides a standard for tagging financial figures, mapping thousands of facts to over 10k US GAAP concepts remains costly and error prone. Existing benchmarks oversimplify this task as flat, single step classification over small subsets of concepts, ignoring the hierarchical semantics of the taxonomy and the structured nature of financial documents. Consequently, these benchmarks fail to evaluate Large Language Models (LLMs) under realistic reporting conditions. To bridge this gap, we introduce FinTagging, the first comprehensive benchmark for structure aware and full scope XBRL tagging. We decompose the complex tagging process into two subtasks: (1) FinNI (Financial Numeric Identification), which extracts entities and types from heterogeneous contexts including text and tables; and (2) FinCL (Financial Concept Linking), which maps extracted entities to the full US GAAP taxonomy. This two stage formulation enables a fair assessment of LLMs' capabilities in numerical reasoning and taxonomy alignment. Evaluating diverse LLMs in zero shot settings reveals that while models generalize well in extraction, they struggle significantly with fine grained concept linking, highlighting critical limitations in domain specific structure aware reasoning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_20650 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FinTagging: Benchmarking LLMs for Extracting and Structuring Financial Information Wang, Yan Qian, Lingfei Peng, Xueqing Ren, Yang Wang, Keyi Han, Yi Feng, Dongji Mo, Fengran Lin, Shengyuan Zhang, Qinchuan He, Kaiwen Luo, Chenri Chen, Jianxing Wu, Junwei Xu, Chen Xu, Ziyang Huang, Jimin Xiong, Guojun Liu, Xiao-Yang Xie, Qianqian Nie, Jian-Yun Computation and Language Artificial Intelligence Computational Engineering, Finance, and Science Accurate interpretation of numerical data in financial reports is critical for markets and regulators. Although XBRL (eXtensible Business Reporting Language) provides a standard for tagging financial figures, mapping thousands of facts to over 10k US GAAP concepts remains costly and error prone. Existing benchmarks oversimplify this task as flat, single step classification over small subsets of concepts, ignoring the hierarchical semantics of the taxonomy and the structured nature of financial documents. Consequently, these benchmarks fail to evaluate Large Language Models (LLMs) under realistic reporting conditions. To bridge this gap, we introduce FinTagging, the first comprehensive benchmark for structure aware and full scope XBRL tagging. We decompose the complex tagging process into two subtasks: (1) FinNI (Financial Numeric Identification), which extracts entities and types from heterogeneous contexts including text and tables; and (2) FinCL (Financial Concept Linking), which maps extracted entities to the full US GAAP taxonomy. This two stage formulation enables a fair assessment of LLMs' capabilities in numerical reasoning and taxonomy alignment. Evaluating diverse LLMs in zero shot settings reveals that while models generalize well in extraction, they struggle significantly with fine grained concept linking, highlighting critical limitations in domain specific structure aware reasoning. |
| title | FinTagging: Benchmarking LLMs for Extracting and Structuring Financial Information |
| topic | Computation and Language Artificial Intelligence Computational Engineering, Finance, and Science |
| url | https://arxiv.org/abs/2505.20650 |