Rationality Check! Benchmarking the Rationality of Large Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908545387069440 |
|---|---|
| author | Zhou, Zhilun Wang, Jing Yi Sukiennik, Nicholas Gao, Chen Xu, Fengli Li, Yong Evans, James |
| author_facet | Zhou, Zhilun Wang, Jing Yi Sukiennik, Nicholas Gao, Chen Xu, Fengli Li, Yong Evans, James |
| contents | Large language models (LLMs), a recent advance in deep learning and machine intelligence, have manifested astonishing capacities, now considered among the most promising for artificial general intelligence. With human-like capabilities, LLMs have been used to simulate humans and serve as AI assistants across many applications. As a result, great concern has arisen about whether and under what circumstances LLMs think and behave like real human agents. Rationality is among the most important concepts in assessing human behavior, both in thinking (i.e., theoretical rationality) and in taking action (i.e., practical rationality). In this work, we propose the first benchmark for evaluating the omnibus rationality of LLMs, covering a wide range of domains and LLMs. The benchmark includes an easy-to-use toolkit, extensive experimental results, and analysis that illuminates where LLMs converge and diverge from idealized human rationality. We believe the benchmark can serve as a foundational tool for both developers and users of LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_14546 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Rationality Check! Benchmarking the Rationality of Large Language Models Zhou, Zhilun Wang, Jing Yi Sukiennik, Nicholas Gao, Chen Xu, Fengli Li, Yong Evans, James Artificial Intelligence Large language models (LLMs), a recent advance in deep learning and machine intelligence, have manifested astonishing capacities, now considered among the most promising for artificial general intelligence. With human-like capabilities, LLMs have been used to simulate humans and serve as AI assistants across many applications. As a result, great concern has arisen about whether and under what circumstances LLMs think and behave like real human agents. Rationality is among the most important concepts in assessing human behavior, both in thinking (i.e., theoretical rationality) and in taking action (i.e., practical rationality). In this work, we propose the first benchmark for evaluating the omnibus rationality of LLMs, covering a wide range of domains and LLMs. The benchmark includes an easy-to-use toolkit, extensive experimental results, and analysis that illuminates where LLMs converge and diverge from idealized human rationality. We believe the benchmark can serve as a foundational tool for both developers and users of LLMs. |
| title | Rationality Check! Benchmarking the Rationality of Large Language Models |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2509.14546 |