Tracing the complexity profiles of different linguistic phenomena through the intrinsic dimension of LLM representations
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908990550573056 |
|---|---|
| author | Baroni, Marco Cheng, Emily de-Dios-Flores, Iria Franzon, Francesca |
| author_facet | Baroni, Marco Cheng, Emily de-Dios-Flores, Iria Franzon, Francesca |
| contents | We explore intrinsic dimension (ID) of LLM representations as a marker of linguistic complexity. Specifically, we test whether ID differences across model layers reflect well-known complexity contrasts established in (psycho)linguistics: coordination vs. subordination, right-branching vs. center-embedding, and unambiguous vs. ambiguous attachment. Our results on six different LLMs show that these contrasts are consistently reflected in ID differences, with more complex phenomena eliciting higher ID profiles. Notably, ID differences emerge at different points across layers for different contrasts, also reaching their peaks at different stages. Further experiments using representational similarity and layer pruning confirm the trends. We conclude that ID is a useful marker of linguistic complexity in LLMs, that it points to similar linguistic processing steps across disparate LLMs, and that it has the potential to differentiate between different types of complexity. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_03779 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Tracing the complexity profiles of different linguistic phenomena through the intrinsic dimension of LLM representations Baroni, Marco Cheng, Emily de-Dios-Flores, Iria Franzon, Francesca Computation and Language We explore intrinsic dimension (ID) of LLM representations as a marker of linguistic complexity. Specifically, we test whether ID differences across model layers reflect well-known complexity contrasts established in (psycho)linguistics: coordination vs. subordination, right-branching vs. center-embedding, and unambiguous vs. ambiguous attachment. Our results on six different LLMs show that these contrasts are consistently reflected in ID differences, with more complex phenomena eliciting higher ID profiles. Notably, ID differences emerge at different points across layers for different contrasts, also reaching their peaks at different stages. Further experiments using representational similarity and layer pruning confirm the trends. We conclude that ID is a useful marker of linguistic complexity in LLMs, that it points to similar linguistic processing steps across disparate LLMs, and that it has the potential to differentiate between different types of complexity. |
| title | Tracing the complexity profiles of different linguistic phenomena through the intrinsic dimension of LLM representations |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2601.03779 |