CREPE: Coordinate-Aware End-to-End Document Parser
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914778563215360 |
|---|---|
| author | Okamoto, Yamato Baek, Youngmin Kim, Geewook Nakao, Ryota Kim, DongHyun Yim, Moon Bin Park, Seunghyun Lee, Bado |
| author_facet | Okamoto, Yamato Baek, Youngmin Kim, Geewook Nakao, Ryota Kim, DongHyun Yim, Moon Bin Park, Seunghyun Lee, Bado |
| contents | In this study, we formulate an OCR-free sequence generation model for visual document understanding (VDU). Our model not only parses text from document images but also extracts the spatial coordinates of the text based on the multi-head architecture. Named as Coordinate-aware End-to-end Document Parser (CREPE), our method uniquely integrates these capabilities by introducing a special token for OCR text, and token-triggered coordinate decoding. We also proposed a weakly-supervised framework for cost-efficient training, requiring only parsing annotations without high-cost coordinate annotations. Our experimental evaluations demonstrate CREPE's state-of-the-art performances on document parsing tasks. Beyond that, CREPE's adaptability is further highlighted by its successful usage in other document understanding tasks such as layout analysis, document visual question answering, and so one. CREPE's abilities including OCR and semantic parsing not only mitigate error propagation issues in existing OCR-dependent methods, it also significantly enhance the functionality of sequence generation models, ushering in a new era for document understanding studies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_00260 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CREPE: Coordinate-Aware End-to-End Document Parser Okamoto, Yamato Baek, Youngmin Kim, Geewook Nakao, Ryota Kim, DongHyun Yim, Moon Bin Park, Seunghyun Lee, Bado Computer Vision and Pattern Recognition In this study, we formulate an OCR-free sequence generation model for visual document understanding (VDU). Our model not only parses text from document images but also extracts the spatial coordinates of the text based on the multi-head architecture. Named as Coordinate-aware End-to-end Document Parser (CREPE), our method uniquely integrates these capabilities by introducing a special token for OCR text, and token-triggered coordinate decoding. We also proposed a weakly-supervised framework for cost-efficient training, requiring only parsing annotations without high-cost coordinate annotations. Our experimental evaluations demonstrate CREPE's state-of-the-art performances on document parsing tasks. Beyond that, CREPE's adaptability is further highlighted by its successful usage in other document understanding tasks such as layout analysis, document visual question answering, and so one. CREPE's abilities including OCR and semantic parsing not only mitigate error propagation issues in existing OCR-dependent methods, it also significantly enhance the functionality of sequence generation models, ushering in a new era for document understanding studies. |
| title | CREPE: Coordinate-Aware End-to-End Document Parser |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2405.00260 |