Hidden Failures in Robustness: Why Supervised Uncertainty Quantification Needs Better Evaluation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915934878302208 |
|---|---|
| author | Stacey, Joe Orgad, Hadas Inui, Kentaro Heinzerling, Benjamin Moosavi, Nafise Sadat |
| author_facet | Stacey, Joe Orgad, Hadas Inui, Kentaro Heinzerling, Benjamin Moosavi, Nafise Sadat |
| contents | Recent work has shown that the hidden states of large language models contain signals useful for uncertainty estimation and hallucination detection, motivating a growing interest in efficient probe-based approaches. Yet it remains unclear how robust existing methods are, and which probe designs provide uncertainty estimates that are reliable under distribution shift. We present a systematic study of supervised uncertainty probes across models, tasks, and OOD settings, training over 2,000 probes while varying the representation layer, feature type, and token aggregation strategy. Our evaluation highlights poor robustness in current methods, particularly in the case of long-form generations. We also find that probe robustness is driven less by architecture and more by the probe inputs. Middle-layer representations generalise more reliably than final-layer hidden states, and aggregating across response tokens is consistently more robust than relying on single-token features. These differences are often largely invisible in-distribution but become more important under distribution shift. Informed by our evaluation, we explore a simple hybrid back-off strategy for improving robustness, arguing that better evaluation is a prerequisite for building more robust probes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_11662 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Hidden Failures in Robustness: Why Supervised Uncertainty Quantification Needs Better Evaluation Stacey, Joe Orgad, Hadas Inui, Kentaro Heinzerling, Benjamin Moosavi, Nafise Sadat Computation and Language I.2.7 Recent work has shown that the hidden states of large language models contain signals useful for uncertainty estimation and hallucination detection, motivating a growing interest in efficient probe-based approaches. Yet it remains unclear how robust existing methods are, and which probe designs provide uncertainty estimates that are reliable under distribution shift. We present a systematic study of supervised uncertainty probes across models, tasks, and OOD settings, training over 2,000 probes while varying the representation layer, feature type, and token aggregation strategy. Our evaluation highlights poor robustness in current methods, particularly in the case of long-form generations. We also find that probe robustness is driven less by architecture and more by the probe inputs. Middle-layer representations generalise more reliably than final-layer hidden states, and aggregating across response tokens is consistently more robust than relying on single-token features. These differences are often largely invisible in-distribution but become more important under distribution shift. Informed by our evaluation, we explore a simple hybrid back-off strategy for improving robustness, arguing that better evaluation is a prerequisite for building more robust probes. |
| title | Hidden Failures in Robustness: Why Supervised Uncertainty Quantification Needs Better Evaluation |
| topic | Computation and Language I.2.7 |
| url | https://arxiv.org/abs/2604.11662 |