Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.04271 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929744738516992 |
|---|---|
| author | van de Sande, Yana Açar, Gunes van Woudenberg, Thabo Larson, Martha |
| author_facet | van de Sande, Yana Açar, Gunes van Woudenberg, Thabo Larson, Martha |
| contents | We study LLM judgments of misinformation expressed with uncertainty. Our experiments study the response of three widely used LLMs (GPT-4o, LlaMA3, DeepSeek-v2) to misinformation propositions that have been verified false and then are transformed into uncertain statements according to an uncertainty typology. Our results show that after transformation, LLMs change their factchecking classification from false to not-false in 25% of the cases. Analysis reveals that the change cannot be explained by predictors to which humans are expected to be sensitive, i.e., modality, linguistic cues, or argumentation strategy. The exception is doxastic transformations, which use linguistic cue phrases such as "It is believed ...".To gain further insight, we prompt the LLM to make another judgment about the transformed misinformation statements that is not related to truth value. Specifically, we study LLM estimates of the frequency with which people make the uncertain statement. We find a small but significant correlation between judgment of fact and estimation of frequency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_04271 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | On Fact and Frequency: LLM Responses to Misinformation Expressed with Uncertainty van de Sande, Yana Açar, Gunes van Woudenberg, Thabo Larson, Martha Computation and Language Computers and Society 91F20 I.2.7; I.2.4; K.4; J.4 We study LLM judgments of misinformation expressed with uncertainty. Our experiments study the response of three widely used LLMs (GPT-4o, LlaMA3, DeepSeek-v2) to misinformation propositions that have been verified false and then are transformed into uncertain statements according to an uncertainty typology. Our results show that after transformation, LLMs change their factchecking classification from false to not-false in 25% of the cases. Analysis reveals that the change cannot be explained by predictors to which humans are expected to be sensitive, i.e., modality, linguistic cues, or argumentation strategy. The exception is doxastic transformations, which use linguistic cue phrases such as "It is believed ...".To gain further insight, we prompt the LLM to make another judgment about the transformed misinformation statements that is not related to truth value. Specifically, we study LLM estimates of the frequency with which people make the uncertain statement. We find a small but significant correlation between judgment of fact and estimation of frequency. |
| title | On Fact and Frequency: LLM Responses to Misinformation Expressed with Uncertainty |
| topic | Computation and Language Computers and Society 91F20 I.2.7; I.2.4; K.4; J.4 |
| url | https://arxiv.org/abs/2503.04271 |