SCITAT: A Question Answering Benchmark for Scientific Tables and Text Covering Diverse Reasoning Types
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912158890065920 |
|---|---|
| author | Zhang, Xuanliang Wang, Dingzirui Wang, Baoxin Dou, Longxu Lu, Xinyuan Xu, Keyan Wu, Dayong Zhu, Qingfu Che, Wanxiang |
| author_facet | Zhang, Xuanliang Wang, Dingzirui Wang, Baoxin Dou, Longxu Lu, Xinyuan Xu, Keyan Wu, Dayong Zhu, Qingfu Che, Wanxiang |
| contents | Scientific question answering (SQA) is an important task aimed at answering questions based on papers. However, current SQA datasets have limited reasoning types and neglect the relevance between tables and text, creating a significant gap with real scenarios. To address these challenges, we propose a QA benchmark for scientific tables and text with diverse reasoning types (SciTaT). To cover more reasoning types, we summarize various reasoning types from real-world questions. To involve both tables and text, we require the questions to incorporate tables and text as much as possible. Based on SciTaT, we propose a strong baseline (CaR), which combines various reasoning methods to address different reasoning types and process tables and text at the same time. CaR brings average improvements of 12.9% over other baselines on SciTaT, validating its effectiveness. Error analysis reveals the challenges of SciTaT, such as complex numerical calculations and domain knowledge. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_11757 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SCITAT: A Question Answering Benchmark for Scientific Tables and Text Covering Diverse Reasoning Types Zhang, Xuanliang Wang, Dingzirui Wang, Baoxin Dou, Longxu Lu, Xinyuan Xu, Keyan Wu, Dayong Zhu, Qingfu Che, Wanxiang Computation and Language Scientific question answering (SQA) is an important task aimed at answering questions based on papers. However, current SQA datasets have limited reasoning types and neglect the relevance between tables and text, creating a significant gap with real scenarios. To address these challenges, we propose a QA benchmark for scientific tables and text with diverse reasoning types (SciTaT). To cover more reasoning types, we summarize various reasoning types from real-world questions. To involve both tables and text, we require the questions to incorporate tables and text as much as possible. Based on SciTaT, we propose a strong baseline (CaR), which combines various reasoning methods to address different reasoning types and process tables and text at the same time. CaR brings average improvements of 12.9% over other baselines on SciTaT, validating its effectiveness. Error analysis reveals the challenges of SciTaT, such as complex numerical calculations and domain knowledge. |
| title | SCITAT: A Question Answering Benchmark for Scientific Tables and Text Covering Diverse Reasoning Types |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2412.11757 |