Exploring the Potential for Large Language Models to Demonstrate Rational Probabilistic Beliefs
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912334196244480 |
|---|---|
| author | Freedman, Gabriel Toni, Francesca |
| author_facet | Freedman, Gabriel Toni, Francesca |
| contents | Advances in the general capabilities of large language models (LLMs) have led to their use for information retrieval, and as components in automated decision systems. A faithful representation of probabilistic reasoning in these models may be essential to ensure trustworthy, explainable and effective performance in these tasks. Despite previous work suggesting that LLMs can perform complex reasoning and well-calibrated uncertainty quantification, we find that current versions of this class of model lack the ability to provide rational and coherent representations of probabilistic beliefs. To demonstrate this, we introduce a novel dataset of claims with indeterminate truth values and apply a number of well-established techniques for uncertainty quantification to measure the ability of LLM's to adhere to fundamental properties of probabilistic reasoning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_13644 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Exploring the Potential for Large Language Models to Demonstrate Rational Probabilistic Beliefs Freedman, Gabriel Toni, Francesca Artificial Intelligence Computation and Language Advances in the general capabilities of large language models (LLMs) have led to their use for information retrieval, and as components in automated decision systems. A faithful representation of probabilistic reasoning in these models may be essential to ensure trustworthy, explainable and effective performance in these tasks. Despite previous work suggesting that LLMs can perform complex reasoning and well-calibrated uncertainty quantification, we find that current versions of this class of model lack the ability to provide rational and coherent representations of probabilistic beliefs. To demonstrate this, we introduce a novel dataset of claims with indeterminate truth values and apply a number of well-established techniques for uncertainty quantification to measure the ability of LLM's to adhere to fundamental properties of probabilistic reasoning. |
| title | Exploring the Potential for Large Language Models to Demonstrate Rational Probabilistic Beliefs |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2504.13644 |