Generics are puzzling. Can language models find the missing piece?
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916525140606976 |
|---|---|
| author | Calderón, Gustavo Cilleruelo Allaway, Emily Haddow, Barry Birch, Alexandra |
| author_facet | Calderón, Gustavo Cilleruelo Allaway, Emily Haddow, Barry Birch, Alexandra |
| contents | Generic sentences express generalisations about the world without explicit quantification. Although generics are central to everyday communication, building a precise semantic framework has proven difficult, in part because speakers use generics to generalise properties with widely different statistical prevalence. In this work, we study the implicit quantification and context-sensitivity of generics by leveraging language models as models of language. We create ConGen, a dataset of 2873 naturally occurring generic and quantified sentences in context, and define p-acceptability, a metric based on surprisal that is sensitive to quantification. Our experiments show generics are more context-sensitive than determiner quantifiers and about 20% of naturally occurring generics we analyze express weak generalisations. We also explore how human biases in stereotypes can be observed in language models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_11318 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Generics are puzzling. Can language models find the missing piece? Calderón, Gustavo Cilleruelo Allaway, Emily Haddow, Barry Birch, Alexandra Computation and Language Artificial Intelligence Generic sentences express generalisations about the world without explicit quantification. Although generics are central to everyday communication, building a precise semantic framework has proven difficult, in part because speakers use generics to generalise properties with widely different statistical prevalence. In this work, we study the implicit quantification and context-sensitivity of generics by leveraging language models as models of language. We create ConGen, a dataset of 2873 naturally occurring generic and quantified sentences in context, and define p-acceptability, a metric based on surprisal that is sensitive to quantification. Our experiments show generics are more context-sensitive than determiner quantifiers and about 20% of naturally occurring generics we analyze express weak generalisations. We also explore how human biases in stereotypes can be observed in language models. |
| title | Generics are puzzling. Can language models find the missing piece? |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2412.11318 |