Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2407.06349 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914959551627264 |
|---|---|
| author | Roberts, Jesse Moore, Kyle Pham, Thao Ewaleifoh, Oseremhen Fisher, Doug |
| author_facet | Roberts, Jesse Moore, Kyle Pham, Thao Ewaleifoh, Oseremhen Fisher, Doug |
| contents | This paper evaluates whether large language models (LLMs) exhibit cognitive fan effects, similar to those discovered by Anderson in humans, after being pre-trained on human textual data. We conduct two sets of in-context recall experiments designed to elicit fan effects. Consistent with human results, we find that LLM recall uncertainty, measured via token probability, is influenced by the fan effect. Our results show that removing uncertainty disrupts the observed effect. The experiments suggest the fan effect is consistent whether the fan value is induced in-context or in the pre-training data. Finally, these findings provide in-silico evidence that fan effects and typicality are expressions of the same phenomena. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_06349 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Large Language Model Recall Uncertainty is Modulated by the Fan Effect Roberts, Jesse Moore, Kyle Pham, Thao Ewaleifoh, Oseremhen Fisher, Doug Computation and Language Artificial Intelligence This paper evaluates whether large language models (LLMs) exhibit cognitive fan effects, similar to those discovered by Anderson in humans, after being pre-trained on human textual data. We conduct two sets of in-context recall experiments designed to elicit fan effects. Consistent with human results, we find that LLM recall uncertainty, measured via token probability, is influenced by the fan effect. Our results show that removing uncertainty disrupts the observed effect. The experiments suggest the fan effect is consistent whether the fan value is induced in-context or in the pre-training data. Finally, these findings provide in-silico evidence that fan effects and typicality are expressions of the same phenomena. |
| title | Large Language Model Recall Uncertainty is Modulated by the Fan Effect |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2407.06349 |