Benchmarking Vision-Language Models on Chinese Ancient Documents: From OCR to Knowledge Reasoning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912584506015744 |
|---|---|
| author | Yu, Haiyang Wu, Yuchuan Shi, Fan Liao, Lei Lu, Jinghui Ge, Xiaodong Wang, Han Zhuo, Minghan Wu, Xuecheng Fei, Xiang Feng, Hao Tang, Guozhi Wang, An-Lan Zhu, Hanshen He, Yangfan Liang, Quanhuan Meng, Liyuan Feng, Chao Huang, Can Tang, Jingqun Li, Bin |
| author_facet | Yu, Haiyang Wu, Yuchuan Shi, Fan Liao, Lei Lu, Jinghui Ge, Xiaodong Wang, Han Zhuo, Minghan Wu, Xuecheng Fei, Xiang Feng, Hao Tang, Guozhi Wang, An-Lan Zhu, Hanshen He, Yangfan Liang, Quanhuan Meng, Liyuan Feng, Chao Huang, Can Tang, Jingqun Li, Bin |
| contents | Chinese ancient documents, invaluable carriers of millennia of Chinese history and culture, hold rich knowledge across diverse fields but face challenges in digitization and understanding, i.e., traditional methods only scan images, while current Vision-Language Models (VLMs) struggle with their visual and linguistic complexity. Existing document benchmarks focus on English printed texts or simplified Chinese, leaving a gap for evaluating VLMs on ancient Chinese documents. To address this, we present AncientDoc, the first benchmark for Chinese ancient documents, designed to assess VLMs from OCR to knowledge reasoning. AncientDoc includes five tasks (page-level OCR, vernacular translation, reasoning-based QA, knowledge-based QA, linguistic variant QA) and covers 14 document types, over 100 books, and about 3,000 pages. Based on AncientDoc, we evaluate mainstream VLMs using multiple metrics, supplemented by a human-aligned large language model for scoring. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_09731 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Benchmarking Vision-Language Models on Chinese Ancient Documents: From OCR to Knowledge Reasoning Yu, Haiyang Wu, Yuchuan Shi, Fan Liao, Lei Lu, Jinghui Ge, Xiaodong Wang, Han Zhuo, Minghan Wu, Xuecheng Fei, Xiang Feng, Hao Tang, Guozhi Wang, An-Lan Zhu, Hanshen He, Yangfan Liang, Quanhuan Meng, Liyuan Feng, Chao Huang, Can Tang, Jingqun Li, Bin Computation and Language Chinese ancient documents, invaluable carriers of millennia of Chinese history and culture, hold rich knowledge across diverse fields but face challenges in digitization and understanding, i.e., traditional methods only scan images, while current Vision-Language Models (VLMs) struggle with their visual and linguistic complexity. Existing document benchmarks focus on English printed texts or simplified Chinese, leaving a gap for evaluating VLMs on ancient Chinese documents. To address this, we present AncientDoc, the first benchmark for Chinese ancient documents, designed to assess VLMs from OCR to knowledge reasoning. AncientDoc includes five tasks (page-level OCR, vernacular translation, reasoning-based QA, knowledge-based QA, linguistic variant QA) and covers 14 document types, over 100 books, and about 3,000 pages. Based on AncientDoc, we evaluate mainstream VLMs using multiple metrics, supplemented by a human-aligned large language model for scoring. |
| title | Benchmarking Vision-Language Models on Chinese Ancient Documents: From OCR to Knowledge Reasoning |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2509.09731 |