HighlightBench: Benchmarking Markup-Driven Table Reasoning in Scientific Documents
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914427175960576 |
|---|---|
| author | Wang, Lexin Liu, Shenghua Wang, Yiwei Cai, Yujun Ge, Yuyao Yao, Jiayu Guo, Jiafeng Cheng, Xueqi |
| author_facet | Wang, Lexin Liu, Shenghua Wang, Yiwei Cai, Yujun Ge, Yuyao Yao, Jiayu Guo, Jiafeng Cheng, Xueqi |
| contents | Visual markups such as highlights, underlines, and bold text are common in table-centric documents. Although multimodal large language models (MLLMs) have made substantial progress in document understanding, their ability to treat such cues as explicit logical directives remains under-explored. More importantly, existing evaluations cannot distinguish whether a model fails to see the markup or fails to reason with it. This creates a key blind spot in assessing markup-conditioned behavior over tables. To address this gap, we introduce HighlightBench, a diagnostic benchmark for markup-driven table understanding that decomposes evaluation into five task families: Markup Grounding, Constrained Retrieval, Local Relations, Aggregation \& Comparison, and Consistency \& Missingness. We further provide a reference pipeline that makes intermediate decisions explicit, enabling reproducible baselines and finer-grained attribution of errors along the perception-to-execution chain. Experiments show that even strong models remain unstable when visual cues must be consistently aligned with symbolic reasoning under structured output constraints. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_26784 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | HighlightBench: Benchmarking Markup-Driven Table Reasoning in Scientific Documents Wang, Lexin Liu, Shenghua Wang, Yiwei Cai, Yujun Ge, Yuyao Yao, Jiayu Guo, Jiafeng Cheng, Xueqi Computer Vision and Pattern Recognition Visual markups such as highlights, underlines, and bold text are common in table-centric documents. Although multimodal large language models (MLLMs) have made substantial progress in document understanding, their ability to treat such cues as explicit logical directives remains under-explored. More importantly, existing evaluations cannot distinguish whether a model fails to see the markup or fails to reason with it. This creates a key blind spot in assessing markup-conditioned behavior over tables. To address this gap, we introduce HighlightBench, a diagnostic benchmark for markup-driven table understanding that decomposes evaluation into five task families: Markup Grounding, Constrained Retrieval, Local Relations, Aggregation \& Comparison, and Consistency \& Missingness. We further provide a reference pipeline that makes intermediate decisions explicit, enabling reproducible baselines and finer-grained attribution of errors along the perception-to-execution chain. Experiments show that even strong models remain unstable when visual cues must be consistently aligned with symbolic reasoning under structured output constraints. |
| title | HighlightBench: Benchmarking Markup-Driven Table Reasoning in Scientific Documents |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.26784 |