Towards Real-World Document Parsing via Realistic Scene Synthesis and Document-Aware Training
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915942819168256 |
|---|---|
| author | Li, Gengluo Lyu, Pengyuan Zhang, Chengquan Shen, Huawen Wu, Liang Wan, Xingyu Zeng, Gangyan Hu, Han Ma, Can Zhou, Yu |
| author_facet | Li, Gengluo Lyu, Pengyuan Zhang, Chengquan Shen, Huawen Wu, Liang Wan, Xingyu Zeng, Gangyan Hu, Han Ma, Can Zhou, Yu |
| contents | Document parsing has recently advanced with multimodal large language models (MLLMs) that directly map document images to structured outputs. Traditional cascaded pipelines depend on precise layout analysis and often fail under casually captured or non-standard conditions. Although end-to-end approaches mitigate this dependency, they still exhibit repetitive, hallucinated, and structurally inconsistent predictions - primarily due to the scarcity of large-scale, high-quality full-page (document-level) end-to-end parsing data and the lack of structure-aware training strategies. To address these challenges, we propose a data-training co-design framework for robust end-to-end document parsing. A Realistic Scene Synthesis strategy constructs large-scale, structurally diverse full-page end-to-end supervision by composing layout templates with rich document elements, while a Document-Aware Training Recipe introduces progressive learning and structure-token optimization to enhance structural fidelity and decoding stability. We further build Wild-OmniDocBench, a benchmark derived from real-world captured documents for robustness evaluation. Integrated into a 1B-parameter MLLM, our method achieves superior accuracy and robustness across both scanned/digital and real-world captured scenarios. All models, data synthesis pipelines, and benchmarks will be publicly released to advance future research in document understanding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_23885 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Towards Real-World Document Parsing via Realistic Scene Synthesis and Document-Aware Training Li, Gengluo Lyu, Pengyuan Zhang, Chengquan Shen, Huawen Wu, Liang Wan, Xingyu Zeng, Gangyan Hu, Han Ma, Can Zhou, Yu Computer Vision and Pattern Recognition Document parsing has recently advanced with multimodal large language models (MLLMs) that directly map document images to structured outputs. Traditional cascaded pipelines depend on precise layout analysis and often fail under casually captured or non-standard conditions. Although end-to-end approaches mitigate this dependency, they still exhibit repetitive, hallucinated, and structurally inconsistent predictions - primarily due to the scarcity of large-scale, high-quality full-page (document-level) end-to-end parsing data and the lack of structure-aware training strategies. To address these challenges, we propose a data-training co-design framework for robust end-to-end document parsing. A Realistic Scene Synthesis strategy constructs large-scale, structurally diverse full-page end-to-end supervision by composing layout templates with rich document elements, while a Document-Aware Training Recipe introduces progressive learning and structure-token optimization to enhance structural fidelity and decoding stability. We further build Wild-OmniDocBench, a benchmark derived from real-world captured documents for robustness evaluation. Integrated into a 1B-parameter MLLM, our method achieves superior accuracy and robustness across both scanned/digital and real-world captured scenarios. All models, data synthesis pipelines, and benchmarks will be publicly released to advance future research in document understanding. |
| title | Towards Real-World Document Parsing via Realistic Scene Synthesis and Document-Aware Training |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.23885 |