Measuring Contextual Informativeness in Child-Directed Text
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916539808088064 |
|---|---|
| author | Valentini, Maria Wright, Téa Marashian, Ali Weber, Jennifer Colunga, Eliana von der Wense, Katharina |
| author_facet | Valentini, Maria Wright, Téa Marashian, Ali Weber, Jennifer Colunga, Eliana von der Wense, Katharina |
| contents | To address an important gap in creating children's stories for vocabulary enrichment, we investigate the automatic evaluation of how well stories convey the semantics of target vocabulary words, a task with substantial implications for generating educational content. We motivate this task, which we call measuring contextual informativeness in children's stories, and provide a formal task definition as well as a dataset for the task. We further propose a method for automating the task using a large language model (LLM). Our experiments show that our approach reaches a Spearman correlation of 0.4983 with human judgments of informativeness, while the strongest baseline only obtains a correlation of 0.3534. An additional analysis shows that the LLM-based approach is able to generalize to measuring contextual informativeness in adult-directed text, on which it also outperforms all baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_17427 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Measuring Contextual Informativeness in Child-Directed Text Valentini, Maria Wright, Téa Marashian, Ali Weber, Jennifer Colunga, Eliana von der Wense, Katharina Computation and Language To address an important gap in creating children's stories for vocabulary enrichment, we investigate the automatic evaluation of how well stories convey the semantics of target vocabulary words, a task with substantial implications for generating educational content. We motivate this task, which we call measuring contextual informativeness in children's stories, and provide a formal task definition as well as a dataset for the task. We further propose a method for automating the task using a large language model (LLM). Our experiments show that our approach reaches a Spearman correlation of 0.4983 with human judgments of informativeness, while the strongest baseline only obtains a correlation of 0.3534. An additional analysis shows that the LLM-based approach is able to generalize to measuring contextual informativeness in adult-directed text, on which it also outperforms all baselines. |
| title | Measuring Contextual Informativeness in Child-Directed Text |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2412.17427 |