Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2405.08497 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917665443938304 |
|---|---|
| author | Knietaite, Agne Allsebrook, Adam Minkov, Anton Tomaszewski, Adam Slinko, Norbert Johnson, Richard Pickard, Thomas Phelps, Dylan Villavicencio, Aline |
| author_facet | Knietaite, Agne Allsebrook, Adam Minkov, Anton Tomaszewski, Adam Slinko, Norbert Johnson, Richard Pickard, Thomas Phelps, Dylan Villavicencio, Aline |
| contents | Compositionality in language models presents a problem when processing idiomatic expressions, as their meaning often cannot be directly derived from their individual parts. Although fine-tuning and other optimization strategies can be used to improve representations of idiomatic expressions, this depends on the availability of relevant data. We present the Noun Compound Synonym Substitution in Books - NCSSB - datasets, which are created by substitution of synonyms of potentially idiomatic English noun compounds in public domain book texts. We explore the trade-off between data quantity and quality when training models for idiomaticity detection, in conjunction with contextual information obtained locally (from the surrounding sentences) or externally (through language resources). Performance on an idiomaticity detection task indicates that dataset quality is a stronger factor for context-enriched models, but that quantity also plays a role in models without context inclusion strategies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_08497 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Is Less More? Quality, Quantity and Context in Idiom Processing with Natural Language Models Knietaite, Agne Allsebrook, Adam Minkov, Anton Tomaszewski, Adam Slinko, Norbert Johnson, Richard Pickard, Thomas Phelps, Dylan Villavicencio, Aline Computation and Language I.2.7 Compositionality in language models presents a problem when processing idiomatic expressions, as their meaning often cannot be directly derived from their individual parts. Although fine-tuning and other optimization strategies can be used to improve representations of idiomatic expressions, this depends on the availability of relevant data. We present the Noun Compound Synonym Substitution in Books - NCSSB - datasets, which are created by substitution of synonyms of potentially idiomatic English noun compounds in public domain book texts. We explore the trade-off between data quantity and quality when training models for idiomaticity detection, in conjunction with contextual information obtained locally (from the surrounding sentences) or externally (through language resources). Performance on an idiomaticity detection task indicates that dataset quality is a stronger factor for context-enriched models, but that quantity also plays a role in models without context inclusion strategies. |
| title | Is Less More? Quality, Quantity and Context in Idiom Processing with Natural Language Models |
| topic | Computation and Language I.2.7 |
| url | https://arxiv.org/abs/2405.08497 |