Do Construction Distributions Shape Formal Language Learning In German BabyLMs?
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909651195396096 |
|---|---|
| author | Bunzeck, Bastian Duran, Daniel Zarrieß, Sina |
| author_facet | Bunzeck, Bastian Duran, Daniel Zarrieß, Sina |
| contents | We analyze the influence of utterance-level construction distributions in German child-directed/child-available speech on the resulting word-level, syntactic and semantic competence (and their underlying learning trajectories) in small LMs, which we train on a novel collection of developmentally plausible language data for German. We find that trajectories are surprisingly robust for markedly different distributions of constructions in the training data, which have little effect on final accuracies and almost no effect on global learning trajectories. While syntax learning benefits from more complex utterances, word-level learning culminates in better scores with more fragmentary utterances. We argue that LMs trained on developmentally plausible data can contribute to debates on how conducive different kinds of linguistic stimuli are to language learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_11593 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Do Construction Distributions Shape Formal Language Learning In German BabyLMs? Bunzeck, Bastian Duran, Daniel Zarrieß, Sina Computation and Language We analyze the influence of utterance-level construction distributions in German child-directed/child-available speech on the resulting word-level, syntactic and semantic competence (and their underlying learning trajectories) in small LMs, which we train on a novel collection of developmentally plausible language data for German. We find that trajectories are surprisingly robust for markedly different distributions of constructions in the training data, which have little effect on final accuracies and almost no effect on global learning trajectories. While syntax learning benefits from more complex utterances, word-level learning culminates in better scores with more fragmentary utterances. We argue that LMs trained on developmentally plausible data can contribute to debates on how conducive different kinds of linguistic stimuli are to language learning. |
| title | Do Construction Distributions Shape Formal Language Learning In German BabyLMs? |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2503.11593 |