Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.13259 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909353119842304 |
|---|---|
| author | Yang, Qiyuan Wang, Pengda Plonsky, Luke D. Oswald, Frederick L. Chen, Hanjie |
| author_facet | Yang, Qiyuan Wang, Pengda Plonsky, Luke D. Oswald, Frederick L. Chen, Hanjie |
| contents | We examine the language capabilities of language models (LMs) from the critical perspective of human language acquisition. Building on classical language development theories, we propose a three-stage framework to assess the abilities of LMs, ranging from preliminary word understanding to complex grammar and complex logical reasoning. Using this framework, we evaluate the generative capacities of LMs using methods from linguistic research. Results indicate that although recent LMs outperform earlier models in overall performance, their developmental trajectory does not strictly follow the path of human language acquisition. Notably, in generation tasks, LMs are more similar to human performance in areas where information is easier to extract from the corpus, such as average word length, clauses, and auxiliary verbs. Newer LMs did not exhibit significant progress in terms of specific dimensions, such as clauses and auxiliary verbs, where the variation across corpora is relatively limited. Register theory offers a plausible explanation for these observations, suggesting that the linguistic features of the training data have a substantial impact on the models' abilities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_13259 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | From Babbling to Fluency: Evaluating the Evolution of Language Models in Terms of Human Language Acquisition Yang, Qiyuan Wang, Pengda Plonsky, Luke D. Oswald, Frederick L. Chen, Hanjie Computation and Language We examine the language capabilities of language models (LMs) from the critical perspective of human language acquisition. Building on classical language development theories, we propose a three-stage framework to assess the abilities of LMs, ranging from preliminary word understanding to complex grammar and complex logical reasoning. Using this framework, we evaluate the generative capacities of LMs using methods from linguistic research. Results indicate that although recent LMs outperform earlier models in overall performance, their developmental trajectory does not strictly follow the path of human language acquisition. Notably, in generation tasks, LMs are more similar to human performance in areas where information is easier to extract from the corpus, such as average word length, clauses, and auxiliary verbs. Newer LMs did not exhibit significant progress in terms of specific dimensions, such as clauses and auxiliary verbs, where the variation across corpora is relatively limited. Register theory offers a plausible explanation for these observations, suggesting that the linguistic features of the training data have a substantial impact on the models' abilities. |
| title | From Babbling to Fluency: Evaluating the Evolution of Language Models in Terms of Human Language Acquisition |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2410.13259 |