From Input Perception to Predictive Insight: Modeling Model Blind Spots Before They Become Errors
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866918147266707456 |
|---|---|
| author | Mi, Maggie Villavicencio, Aline Moosavi, Nafise Sadat |
| author_facet | Mi, Maggie Villavicencio, Aline Moosavi, Nafise Sadat |
| contents | Language models often struggle with idiomatic, figurative, or context-sensitive inputs, not because they produce flawed outputs, but because they misinterpret the input from the outset. We propose an input-only method for anticipating such failures using token-level likelihood features inspired by surprisal and the Uniform Information Density hypothesis. These features capture localized uncertainty in input comprehension and outperform standard baselines across five linguistically challenging datasets. We show that span-localized features improve error detection for larger models, while smaller models benefit from global patterns. Our method requires no access to outputs or hidden activations, offering a lightweight and generalizable approach to pre-generation error prediction. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_20065 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | From Input Perception to Predictive Insight: Modeling Model Blind Spots Before They Become Errors Mi, Maggie Villavicencio, Aline Moosavi, Nafise Sadat Computation and Language Language models often struggle with idiomatic, figurative, or context-sensitive inputs, not because they produce flawed outputs, but because they misinterpret the input from the outset. We propose an input-only method for anticipating such failures using token-level likelihood features inspired by surprisal and the Uniform Information Density hypothesis. These features capture localized uncertainty in input comprehension and outperform standard baselines across five linguistically challenging datasets. We show that span-localized features improve error detection for larger models, while smaller models benefit from global patterns. Our method requires no access to outputs or hidden activations, offering a lightweight and generalizable approach to pre-generation error prediction. |
| title | From Input Perception to Predictive Insight: Modeling Model Blind Spots Before They Become Errors |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2509.20065 |