Emergence of Phonemic, Syntactic, and Semantic Representations in Artificial Neural Networks
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917223856078848 |
|---|---|
| author | Orhan, Pierre Diego-Simón, Pablo Chemla, Emmnanuel Lakretz, Yair Boubenec, Yves King, Jean-Rémi |
| author_facet | Orhan, Pierre Diego-Simón, Pablo Chemla, Emmnanuel Lakretz, Yair Boubenec, Yves King, Jean-Rémi |
| contents | During language acquisition, children successively learn to categorize phonemes, identify words, and combine them with syntax to form new meaning. While the development of this behavior is well characterized, we still lack a unifying computational framework to explain its underlying neural representations. Here, we investigate whether and when phonemic, lexical, and syntactic representations emerge in the activations of artificial neural networks during their training. Our results show that both speech- and text-based models follow a sequence of learning stages: during training, their neural activations successively build subspaces, where the geometry of the neural activations represents phonemic, lexical, and syntactic structure. While this developmental trajectory qualitatively relates to children's, it is quantitatively different: These algorithms indeed require two to four orders of magnitude more data for these neural representations to emerge. Together, these results show conditions under which major stages of language acquisition spontaneously emerge, and hence delineate a promising path to understand the computations underpinning language acquisition. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_18617 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Emergence of Phonemic, Syntactic, and Semantic Representations in Artificial Neural Networks Orhan, Pierre Diego-Simón, Pablo Chemla, Emmnanuel Lakretz, Yair Boubenec, Yves King, Jean-Rémi Artificial Intelligence Computation and Language During language acquisition, children successively learn to categorize phonemes, identify words, and combine them with syntax to form new meaning. While the development of this behavior is well characterized, we still lack a unifying computational framework to explain its underlying neural representations. Here, we investigate whether and when phonemic, lexical, and syntactic representations emerge in the activations of artificial neural networks during their training. Our results show that both speech- and text-based models follow a sequence of learning stages: during training, their neural activations successively build subspaces, where the geometry of the neural activations represents phonemic, lexical, and syntactic structure. While this developmental trajectory qualitatively relates to children's, it is quantitatively different: These algorithms indeed require two to four orders of magnitude more data for these neural representations to emerge. Together, these results show conditions under which major stages of language acquisition spontaneously emerge, and hence delineate a promising path to understand the computations underpinning language acquisition. |
| title | Emergence of Phonemic, Syntactic, and Semantic Representations in Artificial Neural Networks |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2601.18617 |