Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2505.00063 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918029674151936 |
|---|---|
| author | Li, Siqi Shen, Yufan Chen, Xiangnan Chen, Jiayi Ju, Hengwei Duan, Haodong Mao, Song Zhou, Hongbin Zhang, Bo Fu, Bin Cai, Pinlong Wen, Licheng Shi, Botian Liu, Yong Cai, Xinyu Qiao, Yu |
| author_facet | Li, Siqi Shen, Yufan Chen, Xiangnan Chen, Jiayi Ju, Hengwei Duan, Haodong Mao, Song Zhou, Hongbin Zhang, Bo Fu, Bin Cai, Pinlong Wen, Licheng Shi, Botian Liu, Yong Cai, Xinyu Qiao, Yu |
| contents | The rapid advancement of multimodal large language models (MLLMs) has profoundly impacted the document domain, creating a wide array of application scenarios. This progress highlights the need for a comprehensive benchmark to evaluate these models' capabilities across various document-specific tasks. However, existing benchmarks often fail to locate specific model weaknesses or guide systematic improvements. To bridge this gap, we introduce a General Document Intelligence Benchmark (GDI-Bench), featuring 2.3k images across 9 key scenarios and 19 document-specific tasks. By decoupling visual complexity and reasoning complexity, the GDI-Bench structures graded tasks that allow performance assessment by difficulty, aiding in model weakness identification and optimization guidance. We evaluate various open-source and closed-source models on GDI-Bench, conducting decoupled analyses in the visual and reasoning domains, revealing their strengths and weaknesses. To address the diverse tasks and domains in the GDI-Bench, we propose a GDI-Model that mitigates catastrophic forgetting during the supervised fine-tuning (SFT) process through an intelligence-preserving training strategy, thereby reinforcing the inherent weaknesses of the base model. Our model achieves state-of-the-art performance on previous benchmarks and the GDI-Bench. Both our benchmark and models are or will be open-sourced on https://huggingface.co/GDIBench. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_00063 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | GDI-Bench: A Benchmark for General Document Intelligence with Vision and Reasoning Decoupling Li, Siqi Shen, Yufan Chen, Xiangnan Chen, Jiayi Ju, Hengwei Duan, Haodong Mao, Song Zhou, Hongbin Zhang, Bo Fu, Bin Cai, Pinlong Wen, Licheng Shi, Botian Liu, Yong Cai, Xinyu Qiao, Yu Computation and Language Computer Vision and Pattern Recognition The rapid advancement of multimodal large language models (MLLMs) has profoundly impacted the document domain, creating a wide array of application scenarios. This progress highlights the need for a comprehensive benchmark to evaluate these models' capabilities across various document-specific tasks. However, existing benchmarks often fail to locate specific model weaknesses or guide systematic improvements. To bridge this gap, we introduce a General Document Intelligence Benchmark (GDI-Bench), featuring 2.3k images across 9 key scenarios and 19 document-specific tasks. By decoupling visual complexity and reasoning complexity, the GDI-Bench structures graded tasks that allow performance assessment by difficulty, aiding in model weakness identification and optimization guidance. We evaluate various open-source and closed-source models on GDI-Bench, conducting decoupled analyses in the visual and reasoning domains, revealing their strengths and weaknesses. To address the diverse tasks and domains in the GDI-Bench, we propose a GDI-Model that mitigates catastrophic forgetting during the supervised fine-tuning (SFT) process through an intelligence-preserving training strategy, thereby reinforcing the inherent weaknesses of the base model. Our model achieves state-of-the-art performance on previous benchmarks and the GDI-Bench. Both our benchmark and models are or will be open-sourced on https://huggingface.co/GDIBench. |
| title | GDI-Bench: A Benchmark for General Document Intelligence with Vision and Reasoning Decoupling |
| topic | Computation and Language Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.00063 |