ThaiOCRBench: A Task-Diverse Benchmark for Vision-Language Understanding in Thai
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912747777687552 |
|---|---|
| author | Nonesung, Surapon Jaknamon, Teetouch Chaiophat, Sirinya Nitarach, Natapong Wittayasakpan, Chanakan Sirichotedumrong, Warit Na-Thalang, Adisai Pipatanakul, Kunat |
| author_facet | Nonesung, Surapon Jaknamon, Teetouch Chaiophat, Sirinya Nitarach, Natapong Wittayasakpan, Chanakan Sirichotedumrong, Warit Na-Thalang, Adisai Pipatanakul, Kunat |
| contents | We present ThaiOCRBench, the first comprehensive benchmark for evaluating vision-language models (VLMs) on Thai text-rich visual understanding tasks. Despite recent progress in multimodal modeling, existing benchmarks predominantly focus on high-resource languages, leaving Thai underrepresented, especially in tasks requiring document structure understanding. ThaiOCRBench addresses this gap by offering a diverse, human-annotated dataset comprising 2,808 samples across 13 task categories. We evaluate a wide range of state-of-the-art VLMs in a zero-shot setting, spanning both proprietary and open-source systems. Results show a significant performance gap, with proprietary models (e.g., Gemini 2.5 Pro) outperforming open-source counterparts. Notably, fine-grained text recognition and handwritten content extraction exhibit the steepest performance drops among open-source models. Through detailed error analysis, we identify key challenges such as language bias, structural mismatch, and hallucinated content. ThaiOCRBench provides a standardized framework for assessing VLMs in low-resource, script-complex settings, and provides actionable insights for improving Thai-language document understanding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_04479 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ThaiOCRBench: A Task-Diverse Benchmark for Vision-Language Understanding in Thai Nonesung, Surapon Jaknamon, Teetouch Chaiophat, Sirinya Nitarach, Natapong Wittayasakpan, Chanakan Sirichotedumrong, Warit Na-Thalang, Adisai Pipatanakul, Kunat Computation and Language We present ThaiOCRBench, the first comprehensive benchmark for evaluating vision-language models (VLMs) on Thai text-rich visual understanding tasks. Despite recent progress in multimodal modeling, existing benchmarks predominantly focus on high-resource languages, leaving Thai underrepresented, especially in tasks requiring document structure understanding. ThaiOCRBench addresses this gap by offering a diverse, human-annotated dataset comprising 2,808 samples across 13 task categories. We evaluate a wide range of state-of-the-art VLMs in a zero-shot setting, spanning both proprietary and open-source systems. Results show a significant performance gap, with proprietary models (e.g., Gemini 2.5 Pro) outperforming open-source counterparts. Notably, fine-grained text recognition and handwritten content extraction exhibit the steepest performance drops among open-source models. Through detailed error analysis, we identify key challenges such as language bias, structural mismatch, and hallucinated content. ThaiOCRBench provides a standardized framework for assessing VLMs in low-resource, script-complex settings, and provides actionable insights for improving Thai-language document understanding. |
| title | ThaiOCRBench: A Task-Diverse Benchmark for Vision-Language Understanding in Thai |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2511.04479 |