Semantic Concurrency Limits in Large Language Models
Fuente:
arXiv
Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910224096428032 |
|---|---|
| author | Svozil, Karl |
| author_facet | Svozil, Karl |
| contents | High-dimensional embedding spaces can host many semantic directions with small mutual overlap. But small overlaps are not zero: when many directions are jointly active, their residual interference accumulates and limits what a finite readout channel can recover. We formulate this as a distinction between \emph{kinetic capacity} -- what the geometry can host -- and \emph{epistemic accessibility} -- what readout can recover. The two sides are summarized by N < exp(c d_{eff} ε^2) for coexistence and σ_{int} \sim \sqrt{k/d_{eff}} for simultaneous readout. Thus dimension acts not merely as storage capacity but as semantic concurrency bandwidth. On this geometric foundation we propose a separate hypothesis: some polysemous tokens may be organized around stable token-associated hinge directions, with sense information carried by low-dimensional subspaces in the hinge-perpendicular carrier. The capacity/accessibility distinction is the main claim; the hinge hypothesis is a stronger, separately falsifiable empirical proposal. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_13824 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Semantic Concurrency Limits in Large Language Models Svozil, Karl Quantum Physics High-dimensional embedding spaces can host many semantic directions with small mutual overlap. But small overlaps are not zero: when many directions are jointly active, their residual interference accumulates and limits what a finite readout channel can recover. We formulate this as a distinction between \emph{kinetic capacity} -- what the geometry can host -- and \emph{epistemic accessibility} -- what readout can recover. The two sides are summarized by N < exp(c d_{eff} ε^2) for coexistence and σ_{int} \sim \sqrt{k/d_{eff}} for simultaneous readout. Thus dimension acts not merely as storage capacity but as semantic concurrency bandwidth. On this geometric foundation we propose a separate hypothesis: some polysemous tokens may be organized around stable token-associated hinge directions, with sense information carried by low-dimensional subspaces in the hinge-perpendicular carrier. The capacity/accessibility distinction is the main claim; the hinge hypothesis is a stronger, separately falsifiable empirical proposal. |
| title | Semantic Concurrency Limits in Large Language Models |
| topic | Quantum Physics |
| url | https://arxiv.org/abs/2504.13824 |