Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.05488 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916662665543680 |
|---|---|
| author | Khang, Minsoo Jung, Sang Chul Park, Sungrae Hong, Teakgyu |
| author_facet | Khang, Minsoo Jung, Sang Chul Park, Sungrae Hong, Teakgyu |
| contents | Document Key Information Extraction (KIE) is a technology that transforms valuable information in document images into structured data, and it has become an essential function in industrial settings. However, current evaluation metrics of this technology do not accurately reflect the critical attributes of its industrial applications. In this paper, we present KIEval, a novel application-centric evaluation metric for Document KIE models. Unlike prior metrics, KIEval assesses Document KIE models not just on the extraction of individual information (entity) but also of the structured information (grouping). Evaluation of structured information provides assessment of Document KIE models that are more reflective of extracting grouped information from documents in industrial settings. Designed with industrial application in mind, we believe that KIEval can become a standard evaluation metric for developing or applying Document KIE models in practice. The code will be publicly available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_05488 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | KIEval: Evaluation Metric for Document Key Information Extraction Khang, Minsoo Jung, Sang Chul Park, Sungrae Hong, Teakgyu Computation and Language Document Key Information Extraction (KIE) is a technology that transforms valuable information in document images into structured data, and it has become an essential function in industrial settings. However, current evaluation metrics of this technology do not accurately reflect the critical attributes of its industrial applications. In this paper, we present KIEval, a novel application-centric evaluation metric for Document KIE models. Unlike prior metrics, KIEval assesses Document KIE models not just on the extraction of individual information (entity) but also of the structured information (grouping). Evaluation of structured information provides assessment of Document KIE models that are more reflective of extracting grouped information from documents in industrial settings. Designed with industrial application in mind, we believe that KIEval can become a standard evaluation metric for developing or applying Document KIE models in practice. The code will be publicly available. |
| title | KIEval: Evaluation Metric for Document Key Information Extraction |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2503.05488 |