Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2309.10952 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913400434458624 |
|---|---|
| author | Perot, Vincent Kang, Kai Luisier, Florian Su, Guolong Sun, Xiaoyu Boppana, Ramya Sree Wang, Zilong Wang, Zifeng Mu, Jiaqi Zhang, Hao Lee, Chen-Yu Hua, Nan |
| author_facet | Perot, Vincent Kang, Kai Luisier, Florian Su, Guolong Sun, Xiaoyu Boppana, Ramya Sree Wang, Zilong Wang, Zifeng Mu, Jiaqi Zhang, Hao Lee, Chen-Yu Hua, Nan |
| contents | Large Language Models (LLM) have revolutionized Natural Language Processing (NLP), improving state-of-the-art and exhibiting emergent capabilities across various tasks. However, their application in extracting information from visually rich documents, which is at the core of many document processing workflows and involving the extraction of key entities from semi-structured documents, has not yet been successful. The main obstacles to adopting LLMs for this task include the absence of layout encoding within LLMs, which is critical for high quality extraction, and the lack of a grounding mechanism to localize the predicted entities within the document. In this paper, we introduce Language Model-based Document Information Extraction and Localization (LMDX), a methodology to reframe the document information extraction task for a LLM. LMDX enables extraction of singular, repeated, and hierarchical entities, both with and without training data, while providing grounding guarantees and localizing the entities within the document. Finally, we apply LMDX to the PaLM 2-S and Gemini Pro LLMs and evaluate it on VRDU and CORD benchmarks, setting a new state-of-the-art and showing how LMDX enables the creation of high quality, data-efficient parsers. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_10952 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | LMDX: Language Model-based Document Information Extraction and Localization Perot, Vincent Kang, Kai Luisier, Florian Su, Guolong Sun, Xiaoyu Boppana, Ramya Sree Wang, Zilong Wang, Zifeng Mu, Jiaqi Zhang, Hao Lee, Chen-Yu Hua, Nan Computation and Language Artificial Intelligence Machine Learning Large Language Models (LLM) have revolutionized Natural Language Processing (NLP), improving state-of-the-art and exhibiting emergent capabilities across various tasks. However, their application in extracting information from visually rich documents, which is at the core of many document processing workflows and involving the extraction of key entities from semi-structured documents, has not yet been successful. The main obstacles to adopting LLMs for this task include the absence of layout encoding within LLMs, which is critical for high quality extraction, and the lack of a grounding mechanism to localize the predicted entities within the document. In this paper, we introduce Language Model-based Document Information Extraction and Localization (LMDX), a methodology to reframe the document information extraction task for a LLM. LMDX enables extraction of singular, repeated, and hierarchical entities, both with and without training data, while providing grounding guarantees and localizing the entities within the document. Finally, we apply LMDX to the PaLM 2-S and Gemini Pro LLMs and evaluate it on VRDU and CORD benchmarks, setting a new state-of-the-art and showing how LMDX enables the creation of high quality, data-efficient parsers. |
| title | LMDX: Language Model-based Document Information Extraction and Localization |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2309.10952 |