Modeling the language cortex with form-independent and enriched representations of sentence meaning reveals remarkable semantic abstractness
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912623895773184 |
|---|---|
| author | Saha, Shreya Li, Shurui Tuckute, Greta Li, Yuanning Zhang, Ru-Yuan Wehbe, Leila Fedorenko, Evelina Khosla, Meenakshi |
| author_facet | Saha, Shreya Li, Shurui Tuckute, Greta Li, Yuanning Zhang, Ru-Yuan Wehbe, Leila Fedorenko, Evelina Khosla, Meenakshi |
| contents | The human language system represents both linguistic forms and meanings, but the abstractness of the meaning representations remains debated. Here, we searched for abstract representations of meaning in the language cortex by modeling neural responses to sentences using representations from vision and language models. When we generate images corresponding to sentences and extract vision model embeddings, we find that aggregating across multiple generated images yields increasingly accurate predictions of language cortex responses, sometimes rivaling large language models. Similarly, averaging embeddings across multiple paraphrases of a sentence improves prediction accuracy compared to any single paraphrase. Enriching paraphrases with contextual details that may be implicit (e.g., augmenting "I had a pancake" to include details like "maple syrup") further increases prediction accuracy, even surpassing predictions based on the embedding of the original sentence, suggesting that the language system maintains richer and broader semantic representations than language models. Together, these results demonstrate the existence of highly abstract, form-independent meaning representations within the language cortex. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_02354 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Modeling the language cortex with form-independent and enriched representations of sentence meaning reveals remarkable semantic abstractness Saha, Shreya Li, Shurui Tuckute, Greta Li, Yuanning Zhang, Ru-Yuan Wehbe, Leila Fedorenko, Evelina Khosla, Meenakshi Computation and Language Machine Learning The human language system represents both linguistic forms and meanings, but the abstractness of the meaning representations remains debated. Here, we searched for abstract representations of meaning in the language cortex by modeling neural responses to sentences using representations from vision and language models. When we generate images corresponding to sentences and extract vision model embeddings, we find that aggregating across multiple generated images yields increasingly accurate predictions of language cortex responses, sometimes rivaling large language models. Similarly, averaging embeddings across multiple paraphrases of a sentence improves prediction accuracy compared to any single paraphrase. Enriching paraphrases with contextual details that may be implicit (e.g., augmenting "I had a pancake" to include details like "maple syrup") further increases prediction accuracy, even surpassing predictions based on the embedding of the original sentence, suggesting that the language system maintains richer and broader semantic representations than language models. Together, these results demonstrate the existence of highly abstract, form-independent meaning representations within the language cortex. |
| title | Modeling the language cortex with form-independent and enriched representations of sentence meaning reveals remarkable semantic abstractness |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2510.02354 |