Number Representations in LLMs: A Computational Parallel to Human Perception
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929726474420224 |
|---|---|
| author | AlquBoj, H. V. AlQuabeh, Hilal Bojkovic, Velibor Hiraoka, Tatsuya El-Shangiti, Ahmed Oumar Nwadike, Munachiso Inui, Kentaro |
| author_facet | AlquBoj, H. V. AlQuabeh, Hilal Bojkovic, Velibor Hiraoka, Tatsuya El-Shangiti, Ahmed Oumar Nwadike, Munachiso Inui, Kentaro |
| contents | Humans are believed to perceive numbers on a logarithmic mental number line, where smaller values are represented with greater resolution than larger ones. This cognitive bias, supported by neuroscience and behavioral studies, suggests that numerical magnitudes are processed in a sublinear fashion rather than on a uniform linear scale. Inspired by this hypothesis, we investigate whether large language models (LLMs) exhibit a similar logarithmic-like structure in their internal numerical representations. By analyzing how numerical values are encoded across different layers of LLMs, we apply dimensionality reduction techniques such as PCA and PLS followed by geometric regression to uncover latent structures in the learned embeddings. Our findings reveal that the model's numerical representations exhibit sublinear spacing, with distances between values aligning with a logarithmic scale. This suggests that LLMs, much like humans, may encode numbers in a compressed, non-uniform manner. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_16147 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Number Representations in LLMs: A Computational Parallel to Human Perception AlquBoj, H. V. AlQuabeh, Hilal Bojkovic, Velibor Hiraoka, Tatsuya El-Shangiti, Ahmed Oumar Nwadike, Munachiso Inui, Kentaro Computation and Language 68T50 Humans are believed to perceive numbers on a logarithmic mental number line, where smaller values are represented with greater resolution than larger ones. This cognitive bias, supported by neuroscience and behavioral studies, suggests that numerical magnitudes are processed in a sublinear fashion rather than on a uniform linear scale. Inspired by this hypothesis, we investigate whether large language models (LLMs) exhibit a similar logarithmic-like structure in their internal numerical representations. By analyzing how numerical values are encoded across different layers of LLMs, we apply dimensionality reduction techniques such as PCA and PLS followed by geometric regression to uncover latent structures in the learned embeddings. Our findings reveal that the model's numerical representations exhibit sublinear spacing, with distances between values aligning with a logarithmic scale. This suggests that LLMs, much like humans, may encode numbers in a compressed, non-uniform manner. |
| title | Number Representations in LLMs: A Computational Parallel to Human Perception |
| topic | Computation and Language 68T50 |
| url | https://arxiv.org/abs/2502.16147 |