Large Language Model Recall Uncertainty is Modulated by the Fan Effect
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914959551627264 |
|---|---|
| author | Roberts, Jesse Moore, Kyle Pham, Thao Ewaleifoh, Oseremhen Fisher, Doug |
| author_facet | Roberts, Jesse Moore, Kyle Pham, Thao Ewaleifoh, Oseremhen Fisher, Doug |
| contents | This paper evaluates whether large language models (LLMs) exhibit cognitive fan effects, similar to those discovered by Anderson in humans, after being pre-trained on human textual data. We conduct two sets of in-context recall experiments designed to elicit fan effects. Consistent with human results, we find that LLM recall uncertainty, measured via token probability, is influenced by the fan effect. Our results show that removing uncertainty disrupts the observed effect. The experiments suggest the fan effect is consistent whether the fan value is induced in-context or in the pre-training data. Finally, these findings provide in-silico evidence that fan effects and typicality are expressions of the same phenomena. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_06349 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Large Language Model Recall Uncertainty is Modulated by the Fan Effect Roberts, Jesse Moore, Kyle Pham, Thao Ewaleifoh, Oseremhen Fisher, Doug Computation and Language Artificial Intelligence This paper evaluates whether large language models (LLMs) exhibit cognitive fan effects, similar to those discovered by Anderson in humans, after being pre-trained on human textual data. We conduct two sets of in-context recall experiments designed to elicit fan effects. Consistent with human results, we find that LLM recall uncertainty, measured via token probability, is influenced by the fan effect. Our results show that removing uncertainty disrupts the observed effect. The experiments suggest the fan effect is consistent whether the fan value is induced in-context or in the pre-training data. Finally, these findings provide in-silico evidence that fan effects and typicality are expressions of the same phenomena. |
| title | Large Language Model Recall Uncertainty is Modulated by the Fan Effect |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2407.06349 |