Language Model Probabilities are Not Calibrated in Numeric Contexts
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916643919101952 |
|---|---|
| author | Lovering, Charles Krumdick, Michael Lai, Viet Dac Ebner, Seth Kumar, Nilesh Reddy, Varshini Koncel-Kedziorski, Rik Tanner, Chris |
| author_facet | Lovering, Charles Krumdick, Michael Lai, Viet Dac Ebner, Seth Kumar, Nilesh Reddy, Varshini Koncel-Kedziorski, Rik Tanner, Chris |
| contents | Some statements have one well-defined continuation (e.g., "the Eiffel Tower is in [Paris]"), whereas others have a natural distribution over multiple options (e.g., "the weighted coin flip was [Heads/Tails].") We argue that language model (LM) outputs should capture these natural distributions. Our work specifically tests whether LM output probabilities are calibrated to numeric information within their textual contexts. For example, if the context (the prompt) concerns two equally likely options (e.g., heads or tails for a fair coin), the LM output probabilities should also be equal. Likewise, in a context with nonuniformly likely events (e.g., rolling a pair with two dice) an LM should output proportionate probabilities. However, we find that even in simple settings, the best LMs (1) are poorly calibrated and (2) have systematic biases: artifacts like word identity, word order, and word frequency all impact calibration. For example, gpt-4o-mini often picks the first of two options presented in the prompt regardless of the options' implied likelihoods, whereas Llama-3.1-8B picks the second. Models do not allocate probability mass among valid options in a calibrated manner. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_16007 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Language Model Probabilities are Not Calibrated in Numeric Contexts Lovering, Charles Krumdick, Michael Lai, Viet Dac Ebner, Seth Kumar, Nilesh Reddy, Varshini Koncel-Kedziorski, Rik Tanner, Chris Artificial Intelligence Some statements have one well-defined continuation (e.g., "the Eiffel Tower is in [Paris]"), whereas others have a natural distribution over multiple options (e.g., "the weighted coin flip was [Heads/Tails].") We argue that language model (LM) outputs should capture these natural distributions. Our work specifically tests whether LM output probabilities are calibrated to numeric information within their textual contexts. For example, if the context (the prompt) concerns two equally likely options (e.g., heads or tails for a fair coin), the LM output probabilities should also be equal. Likewise, in a context with nonuniformly likely events (e.g., rolling a pair with two dice) an LM should output proportionate probabilities. However, we find that even in simple settings, the best LMs (1) are poorly calibrated and (2) have systematic biases: artifacts like word identity, word order, and word frequency all impact calibration. For example, gpt-4o-mini often picks the first of two options presented in the prompt regardless of the options' implied likelihoods, whereas Llama-3.1-8B picks the second. Models do not allocate probability mass among valid options in a calibrated manner. |
| title | Language Model Probabilities are Not Calibrated in Numeric Contexts |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2410.16007 |