Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.11015 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909624486068224 |
|---|---|
| author | Wang, An-Lan Tang, Jingqun Lei, Liao Feng, Hao Liu, Qi Fei, Xiang Lu, Jinghui Wang, Han Liu, Weiwei Liu, Hao Liu, Yuliang Bai, Xiang Huang, Can |
| author_facet | Wang, An-Lan Tang, Jingqun Lei, Liao Feng, Hao Liu, Qi Fei, Xiang Lu, Jinghui Wang, Han Liu, Weiwei Liu, Hao Liu, Yuliang Bai, Xiang Huang, Can |
| contents | The rapid advancements in Multimodal Large Language Models (MLLMs) have significantly enhanced capabilities in Document Understanding. However, prevailing benchmarks like DocVQA and ChartQA predominantly comprise \textit{scanned or digital} documents, inadequately reflecting the intricate challenges posed by diverse real-world scenarios, such as variable illumination and physical distortions. This paper introduces WildDoc, the inaugural benchmark designed specifically for assessing document understanding in natural environments. WildDoc incorporates a diverse set of manually captured document images reflecting real-world conditions and leverages document sources from established benchmarks to facilitate comprehensive comparisons with digital or scanned documents. Further, to rigorously evaluate model robustness, each document is captured four times under different conditions. Evaluations of state-of-the-art MLLMs on WildDoc expose substantial performance declines and underscore the models' inadequate robustness compared to traditional benchmarks, highlighting the unique challenges posed by real-world document understanding. Our project homepage is available at https://bytedance.github.io/WildDoc. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_11015 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | WildDoc: How Far Are We from Achieving Comprehensive and Robust Document Understanding in the Wild? Wang, An-Lan Tang, Jingqun Lei, Liao Feng, Hao Liu, Qi Fei, Xiang Lu, Jinghui Wang, Han Liu, Weiwei Liu, Hao Liu, Yuliang Bai, Xiang Huang, Can Computer Vision and Pattern Recognition The rapid advancements in Multimodal Large Language Models (MLLMs) have significantly enhanced capabilities in Document Understanding. However, prevailing benchmarks like DocVQA and ChartQA predominantly comprise \textit{scanned or digital} documents, inadequately reflecting the intricate challenges posed by diverse real-world scenarios, such as variable illumination and physical distortions. This paper introduces WildDoc, the inaugural benchmark designed specifically for assessing document understanding in natural environments. WildDoc incorporates a diverse set of manually captured document images reflecting real-world conditions and leverages document sources from established benchmarks to facilitate comprehensive comparisons with digital or scanned documents. Further, to rigorously evaluate model robustness, each document is captured four times under different conditions. Evaluations of state-of-the-art MLLMs on WildDoc expose substantial performance declines and underscore the models' inadequate robustness compared to traditional benchmarks, highlighting the unique challenges posed by real-world document understanding. Our project homepage is available at https://bytedance.github.io/WildDoc. |
| title | WildDoc: How Far Are We from Achieving Comprehensive and Robust Document Understanding in the Wild? |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.11015 |