Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2507.14675 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913949873602560 |
|---|---|
| author | Duan, Yuchen Chen, Zhe Hu, Yusong Wang, Weiyun Ye, Shenglong Shi, Botian Lu, Lewei Hou, Qibin Lu, Tong Li, Hongsheng Dai, Jifeng Wang, Wenhai |
| author_facet | Duan, Yuchen Chen, Zhe Hu, Yusong Wang, Weiyun Ye, Shenglong Shi, Botian Lu, Lewei Hou, Qibin Lu, Tong Li, Hongsheng Dai, Jifeng Wang, Wenhai |
| contents | Despite significant progress in multimodal large language models (MLLMs), their performance on complex, multi-page document comprehension remains inadequate, largely due to the lack of high-quality, document-level datasets. While current retrieval-augmented generation (RAG) methods offer partial solutions, they suffer from issues, such as fragmented retrieval contexts, multi-stage error accumulation, and extra time costs of retrieval. In this work, we present a high-quality document-level dataset, Doc-750K, designed to support in-depth understanding of multimodal documents. This dataset includes diverse document structures, extensive cross-page dependencies, and real question-answer pairs derived from the original documents. Building on the dataset, we develop a native multimodal model, Docopilot, which can accurately handle document-level dependencies without relying on RAG. Experiments demonstrate that Docopilot achieves superior coherence, accuracy, and efficiency in document understanding tasks and multi-turn interactions, setting a new baseline for document-level multimodal understanding. Data, code, and models are released at https://github.com/OpenGVLab/Docopilot |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_14675 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Docopilot: Improving Multimodal Models for Document-Level Understanding Duan, Yuchen Chen, Zhe Hu, Yusong Wang, Weiyun Ye, Shenglong Shi, Botian Lu, Lewei Hou, Qibin Lu, Tong Li, Hongsheng Dai, Jifeng Wang, Wenhai Computer Vision and Pattern Recognition Computation and Language Despite significant progress in multimodal large language models (MLLMs), their performance on complex, multi-page document comprehension remains inadequate, largely due to the lack of high-quality, document-level datasets. While current retrieval-augmented generation (RAG) methods offer partial solutions, they suffer from issues, such as fragmented retrieval contexts, multi-stage error accumulation, and extra time costs of retrieval. In this work, we present a high-quality document-level dataset, Doc-750K, designed to support in-depth understanding of multimodal documents. This dataset includes diverse document structures, extensive cross-page dependencies, and real question-answer pairs derived from the original documents. Building on the dataset, we develop a native multimodal model, Docopilot, which can accurately handle document-level dependencies without relying on RAG. Experiments demonstrate that Docopilot achieves superior coherence, accuracy, and efficiency in document understanding tasks and multi-turn interactions, setting a new baseline for document-level multimodal understanding. Data, code, and models are released at https://github.com/OpenGVLab/Docopilot |
| title | Docopilot: Improving Multimodal Models for Document-Level Understanding |
| topic | Computer Vision and Pattern Recognition Computation and Language |
| url | https://arxiv.org/abs/2507.14675 |