ViBERTgrid BiLSTM-CRF: Multimodal Key Information Extraction from Unstructured Financial Documents
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913513485631488 |
|---|---|
| author | Pala, Furkan Akpınar, Mehmet Yasin Deniz, Onur Eryiğit, Gülşen |
| author_facet | Pala, Furkan Akpınar, Mehmet Yasin Deniz, Onur Eryiğit, Gülşen |
| contents | Multimodal key information extraction (KIE) models have been studied extensively on semi-structured documents. However, their investigation on unstructured documents is an emerging research topic. The paper presents an approach to adapt a multimodal transformer (i.e., ViBERTgrid previously explored on semi-structured documents) for unstructured financial documents, by incorporating a BiLSTM-CRF layer. The proposed ViBERTgrid BiLSTM-CRF model demonstrates a significant improvement in performance (up to 2 percentage points) on named entity recognition from unstructured documents in financial domain, while maintaining its KIE performance on semi-structured documents. As an additional contribution, we publicly released token-level annotations for the SROIE dataset in order to pave the way for its use in multimodal sequence labeling models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_15004 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ViBERTgrid BiLSTM-CRF: Multimodal Key Information Extraction from Unstructured Financial Documents Pala, Furkan Akpınar, Mehmet Yasin Deniz, Onur Eryiğit, Gülşen Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Information Retrieval Multimodal key information extraction (KIE) models have been studied extensively on semi-structured documents. However, their investigation on unstructured documents is an emerging research topic. The paper presents an approach to adapt a multimodal transformer (i.e., ViBERTgrid previously explored on semi-structured documents) for unstructured financial documents, by incorporating a BiLSTM-CRF layer. The proposed ViBERTgrid BiLSTM-CRF model demonstrates a significant improvement in performance (up to 2 percentage points) on named entity recognition from unstructured documents in financial domain, while maintaining its KIE performance on semi-structured documents. As an additional contribution, we publicly released token-level annotations for the SROIE dataset in order to pave the way for its use in multimodal sequence labeling models. |
| title | ViBERTgrid BiLSTM-CRF: Multimodal Key Information Extraction from Unstructured Financial Documents |
| topic | Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Information Retrieval |
| url | https://arxiv.org/abs/2409.15004 |