Is Large Language Model Performance on Reasoning Tasks Impacted by Different Ways Questions Are Asked?
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917441259438080 |
|---|---|
| author | Song, Seok Hwan Chakraborty, Mohna Li, Qi Tavanapong, Wallapak |
| author_facet | Song, Seok Hwan Chakraborty, Mohna Li, Qi Tavanapong, Wallapak |
| contents | Large Language Models (LLMs) have been evaluated using diverse question types, e.g., multiple-choice, true/false, and short/long answers. This study answers an unexplored question about the impact of different question types on LLM accuracy on reasoning tasks. We investigate the performance of five LLMs on three different types of questions using quantitative and deductive reasoning tasks. The performance metrics include accuracy in the reasoning steps and choosing the final answer. Key Findings: (1) Significant differences exist in LLM performance across different question types. (2) Reasoning accuracy does not necessarily correlate with the final selection accuracy. (3) The number of options and the choice of words, influence LLM performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_15707 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Is Large Language Model Performance on Reasoning Tasks Impacted by Different Ways Questions Are Asked? Song, Seok Hwan Chakraborty, Mohna Li, Qi Tavanapong, Wallapak Computation and Language Artificial Intelligence Large Language Models (LLMs) have been evaluated using diverse question types, e.g., multiple-choice, true/false, and short/long answers. This study answers an unexplored question about the impact of different question types on LLM accuracy on reasoning tasks. We investigate the performance of five LLMs on three different types of questions using quantitative and deductive reasoning tasks. The performance metrics include accuracy in the reasoning steps and choosing the final answer. Key Findings: (1) Significant differences exist in LLM performance across different question types. (2) Reasoning accuracy does not necessarily correlate with the final selection accuracy. (3) The number of options and the choice of words, influence LLM performance. |
| title | Is Large Language Model Performance on Reasoning Tasks Impacted by Different Ways Questions Are Asked? |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2507.15707 |