Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.10739 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909808187146240 |
|---|---|
| author | Pournemat, Mobina Rezaei, Keivan Sriramanan, Gaurang Zarei, Arman Fu, Jiaxiang Wang, Yang Eghbalzadeh, Hamid Feizi, Soheil |
| author_facet | Pournemat, Mobina Rezaei, Keivan Sriramanan, Gaurang Zarei, Arman Fu, Jiaxiang Wang, Yang Eghbalzadeh, Hamid Feizi, Soheil |
| contents | Despite widespread success in language understanding and generation, large language models (LLMs) exhibit unclear and often inconsistent behavior when faced with tasks that require probabilistic reasoning. In this work, we present the first comprehensive study of the reasoning capabilities of LLMs over explicit discrete probability distributions. Given observations from a probability distribution, we evaluate models on three carefully designed tasks, mode identification, maximum likelihood estimation, and sample generation, by prompting them to provide responses to queries about either the joint distribution or its conditionals. These tasks thus probe a range of probabilistic skills, including frequency analysis, marginalization, and generative behavior. Through comprehensive empirical evaluations, we demonstrate that there exists a clear performance gap between smaller and larger models, with the latter demonstrating stronger inference and surprising capabilities in sample generation. Furthermore, our investigations reveal notable limitations, including sensitivity to variations in the notation utilized to represent probabilistic outcomes and performance degradation of over 60% as context length increases. Together, our results provide a detailed understanding of the probabilistic reasoning abilities of LLMs and identify key directions for future improvement. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_10739 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Reasoning Under Uncertainty: Exploring Probabilistic Reasoning Capabilities of LLMs Pournemat, Mobina Rezaei, Keivan Sriramanan, Gaurang Zarei, Arman Fu, Jiaxiang Wang, Yang Eghbalzadeh, Hamid Feizi, Soheil Computation and Language Despite widespread success in language understanding and generation, large language models (LLMs) exhibit unclear and often inconsistent behavior when faced with tasks that require probabilistic reasoning. In this work, we present the first comprehensive study of the reasoning capabilities of LLMs over explicit discrete probability distributions. Given observations from a probability distribution, we evaluate models on three carefully designed tasks, mode identification, maximum likelihood estimation, and sample generation, by prompting them to provide responses to queries about either the joint distribution or its conditionals. These tasks thus probe a range of probabilistic skills, including frequency analysis, marginalization, and generative behavior. Through comprehensive empirical evaluations, we demonstrate that there exists a clear performance gap between smaller and larger models, with the latter demonstrating stronger inference and surprising capabilities in sample generation. Furthermore, our investigations reveal notable limitations, including sensitivity to variations in the notation utilized to represent probabilistic outcomes and performance degradation of over 60% as context length increases. Together, our results provide a detailed understanding of the probabilistic reasoning abilities of LLMs and identify key directions for future improvement. |
| title | Reasoning Under Uncertainty: Exploring Probabilistic Reasoning Capabilities of LLMs |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2509.10739 |