Do Language Models Know Theo Has a Wife? Investigating the Proviso Problem
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917324965019648 |
|---|---|
| author | Azin, Tara Dumitrescu, Daniel Inkpen, Diana Singh, Raj |
| author_facet | Azin, Tara Dumitrescu, Daniel Inkpen, Diana Singh, Raj |
| contents | We investigate how language models handle the proviso problem, an unresolved issue in pragmatics where presuppositions in conditional sentences diverge between theoretical and human interpretations. We reformulate this phenomenon as a Natural Language Inference task and introduce a diagnostic dataset designed to probe presupposition projection in conditionals. We evaluate RoBERTa, DeBERTa, LLaMA, and Gemma using explainability analyses. The results show that models broadly align with human judgments but rely on shallow pattern matching rather than semantic or pragmatic reasoning. Our work provides the first computational evaluation framework for the proviso problem and highlights the need for diagnostic, multi-method approaches to assess pragmatic competence and context-dependent meaning in language models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_08358 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Do Language Models Know Theo Has a Wife? Investigating the Proviso Problem Azin, Tara Dumitrescu, Daniel Inkpen, Diana Singh, Raj Computation and Language We investigate how language models handle the proviso problem, an unresolved issue in pragmatics where presuppositions in conditional sentences diverge between theoretical and human interpretations. We reformulate this phenomenon as a Natural Language Inference task and introduce a diagnostic dataset designed to probe presupposition projection in conditionals. We evaluate RoBERTa, DeBERTa, LLaMA, and Gemma using explainability analyses. The results show that models broadly align with human judgments but rely on shallow pattern matching rather than semantic or pragmatic reasoning. Our work provides the first computational evaluation framework for the proviso problem and highlights the need for diagnostic, multi-method approaches to assess pragmatic competence and context-dependent meaning in language models. |
| title | Do Language Models Know Theo Has a Wife? Investigating the Proviso Problem |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2603.08358 |