Investigating Human-Aligned Large Language Model Uncertainty
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916653794590720 |
|---|---|
| author | Moore, Kyle Roberts, Jesse Watson, Daryl Wisniewski, Pamela |
| author_facet | Moore, Kyle Roberts, Jesse Watson, Daryl Wisniewski, Pamela |
| contents | Recent work has sought to quantify large language model uncertainty to facilitate model control and modulate user trust. Previous works focus on measures of uncertainty that are theoretically grounded or reflect the average overt behavior of the model. In this work, we investigate a variety of uncertainty measures, in order to identify measures that correlate with human group-level uncertainty. We find that Bayesian measures and a variation on entropy measures, top-k entropy, tend to agree with human behavior as a function of model size. We find that some strong measures decrease in human-similarity with model size, but, by multiple linear regression, we find that combining multiple uncertainty measures provide comparable human-alignment with reduced size-dependency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_12528 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Investigating Human-Aligned Large Language Model Uncertainty Moore, Kyle Roberts, Jesse Watson, Daryl Wisniewski, Pamela Computation and Language Recent work has sought to quantify large language model uncertainty to facilitate model control and modulate user trust. Previous works focus on measures of uncertainty that are theoretically grounded or reflect the average overt behavior of the model. In this work, we investigate a variety of uncertainty measures, in order to identify measures that correlate with human group-level uncertainty. We find that Bayesian measures and a variation on entropy measures, top-k entropy, tend to agree with human behavior as a function of model size. We find that some strong measures decrease in human-similarity with model size, but, by multiple linear regression, we find that combining multiple uncertainty measures provide comparable human-alignment with reduced size-dependency. |
| title | Investigating Human-Aligned Large Language Model Uncertainty |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2503.12528 |